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ABSTRACT 

This Study identified and defined 30 structural 
variables related to problem solving. Four dependent variables (only 
one, percent correct, is analyzed in this report) were derived from 
data retained by the computer from 172 arithmetic word problems 
presented to and solved by junior college students on IBM 2741 
computer terminals. The structural and dependent variables were 
combined into a stepwise regression analysis in order to predict 
problem solving difficulty. The order of importance of the 30 
structural variables was determined, and the goodness of fit of the 
model was analyzed by using F and Chi-square tests. Results showed 
that the variable "memory 2" (recall) was highly significant for 
predicting the difficulty of word problems and accounted for if7 
percent of the variance. (DT) 
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PREFACE 

This paper represents a larger study being conducted by the author at 
Golden West College, Huntington Beach, California. Since several parts 
here were used from almost directly that study, the p-esent paper is rather 
sizable. The reader who wishes to obtain the essence of the paper is invited 
to skip to Chapter III, page 29. Furthermore, Appenaices 2 and 3 are related 
to the study in that they describe some of the facilities at the Coast 
Community College District that motivated the study. The reader interested 
in CAI, and CAI related research will find the materials in these two Appendices 
quite interesting. Presently, there are more than 600 CAI units of instruction 
available to students at Golden West College and Orange Coast CoUege, in the 
Coast District. Inquiries should be directed to the author, at Golden West 
College, or to Dr. Bernard Luskin, Office of Educational Development, Coast 
Community College District, Costa Mesa, California. 
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CHAPTER T 



INTRODUCTION 



Problem solving has fascinated research workers for a long time. 
Volumes have been written on the subject. Formal theories and rules of 
thumb have been put forth. Yet, our understanding of the process is still 
not very clear. And, in an increasingly complex society such as ours, 
the understanding of the problem solving process is of tantamount 
importance to the education of the young. The present study reflects an 
effort to contribute to such understanding. 

This study was concerned with one facet of problem solving in 
mathematics education: word-problem solving difficulty. Past research in 
this area was divided into three broad categories based on the type of 
predictor variables used: (a) student variables, (b; presentation 
variables, and (c; structural variables. This study was of the third type. 

In structural variables studies we assume the existence of factors in 
the structure of arithmetic word problems that contribute to the difficulty 
level of the problems. This leads to the identification, classification, 
and definition of a set of factors hereby called structural variables: 



where each xj represents a particular structural characteristic of 



a verbal arithmetic problem. Given n problems, each problem has a 
set of structural variables, thus we may rewrite the set X as: 

Examples of structural variables are the number of words in the problem, 
the noun to verb ratio, and the number of different operations required 
to achieve a solution. 

Once the set X has been identified and defined, each element xj of X, 
is paired vjith a number a j , an element of the set A = -^^o^Ol., • ■ • , ck,^\ in 
such a x^ay that aj reflects the relative importance of in predicting 
problem solving difficulty. Thus, if we hold all other variables 
constant, a unit increp.Ge in y.j cause a relative increase of ai units, 

in problem difficulty. The set A can be derived by any number of statistical 
regression techniques which fall under the category of regression. In this 
study* the set A is derived by linear regression — in particular, stepwise 
linear regression. 

Suppes>I^an, and Jerman(l966) first introduced linear regression as 
a model for predicting difficulty in arithmetic problems. Subsequently, 
Suppes, Loftus, and Jerman (1969), Loftus (1969). and Jerman (1971) ex- 
tended this model to arithmetic word problems. The present study followed 
this path closely, adding some innovations. A major extention planned in 
a more comprehensive study, of which this paper is only a part, is the 
application of canonical correlation and discriminant; analysis to the 
response data. 

STATEMENT OF THE PROBLEM 

This study identified and defined 30 structural variables. Four 
dependent variables (only one will be analyzed here) were derived from 



data trapped by the computer from 172 arithmetic word problems presented 
to and solved by junior college students on IBM 2741 computer terminals. 
The structural and dependent variables were combined into a $te|>vrse re- 
gression analysis in order to predict problem solving difficulty. 
The objective of this study was to: 

a. determine a set of weighs to be assigned to set X, and to 
determine the order of importance of the elements in X; 

b. analyze the goodness of the fit of the niodel by using 
F and Chi-squared tests. 



SIGNIFICANCE OF STUDY 

In California, as well as in many other states, the junior college 
typifies the ideal in American education — free education for all. The re- 
sult of this precept of the junior college, known as the open door policy is 
a widely diversified student population. The most outstanding diversity 
is the academic background of the student population. 

The mathematics curriculum at the junior college reflects this 
diversity by offering courses which range from sixth-grade arithmetic to 
fourth semester calculus; from mathematics for prospective elementary 
school teachers to probability and statistics; and from "tech math" to 
computer programming and elementary numerical analysis. 

These students, indeed, have individual differences. Besides the 
"usual" individual differences they share with their university peers 
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(such as I.Q., attitude, and socio-economic background) junior college 
students differ in past and present academic performance. Whereas one 
student does B work in second semester calculus, another struggles 
through grade-school arithmetic—having difficulty with- problems such as: 
"2 7 is what percent of 30?" 

This study dealt with the latter type of students; for of all the 
subpopulations, as divided by the junior college mathematics curriculum, 
those enrolled in the arithmetic course are perhaps the most pedagogically 
curious. This set of students is separated from the rest by a score of 
four of less on the mathematics part of the SCAT*. Such a score implies 
that after twelve years of formal schooling these students cannot perform 
at a sixth-grade level of mathematics, or, at the very best, seventh-grade 
level Liathematics. 

In the Coast Community College District, where this study was con- 
ducted, and at other jimior colleges, a minimum proficiency in arithmetic 
(SCAT score 4 at Coast) is required for the attainment of the Associate 
of Arts Degree. Thus, these students are forced through yet another try 
at arithmetic—an experience which is usually relished neither by them, 
nor by even the most concerned and conscientious of mathematics faculties. 
This problem, well known to secondary school and junior college mathematics 
educators, has recently come under scrutiny of the Mathematical Association 
of America, an organization of professional mathemticians heretofore in- 



School and College Achievemeut Test. The norms used for arriving at 
the cut-off score of four are local. Junior college students at Orange 
Coast College, Costa Mesa, California, were used to establish these norms. 
For national norms a score of four is equivalent to the 23 percentile. 
SCAT scores range from 0 to 10. 



terested solely in the college and college preparatory mathematics 
curriculum. In a recent publication of the MAA*s Committee of the 
Undergraduate Program in Mathematics (CUPM)* a panel of mathematicians 
described the problem as follows: 

Many students in basic mathematics courses have seen this subject 
matter in elementary and high school without apparent success in 
learning it there* It is often the case that a second exposure to 
essentially the same material similarly organized, is no more success- 
ful even though an attempt is sometimes made to present the subject 
in a more "modem" manner. 

There are certainly many complex reasons for this state of affairs. 
Some of these may be psychological and sociological and may require 
the work of learning theorists and others trained in the social sciences 
in order to lessen their influence... Nonetheless, we believe that it 
deserves very serious consideration by the mathematical community and 
hope that many different kinds of institutions will find our suggestions, 
wholly or in part, of good use when dealing with the type of students 
described. 

The commlttee^s reference to a "modem" manner is obviously directed 
to the "new math" movement, which has not been as successful with low 
achievers in mathematics as it has with the more mathematically talented 
students. However, the committee^s jab at novelty of presentation goes 
beyond the modem math movement. For instance, much effort has recently 
been placed in multimedia approaches to leaming. Schools are presenting 
segments and whole courses through such media as video-tapes, audio-tapes, 
sound-on-slides, computer-assisted instruction, and combinations of these. 
The guiding principle behind this effort is the individualization of in- 
struction—a striving toward that seemingly magical, Socratic ratio of one- 
to-one: one student to one teacher. Though it is logical for educators to 
use the fruits of technology to try to improve leaming, it is not logical 



* Boas/Ralph (Sd.), "A Course in Basic Mathematics for Colleges." 
CUPM, 1971. ^ ' 
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that they expect learning will Indeed he Improved. Unfortunately, educators 
do not, or do not want to recognize this simple fact; topics students find 
difficult In the classroom environment are the same ones they find difficult 
in the environment of other media (Fey, 1969). 

The apparent failure of these innovative methods to improve Instruction 
significantly cannot be blamed on any single characteristic. But one fault 
in the system presently is that the curriculum has not been properly adapted 
to the media. If an innovator uses the standard textbook curriculum in the 
new. Innovative media, he cannot really expect improvement of Instruction. 

Groen and Atkinson (1966) called for such adaptations by offering, 
among others, a Dynamic Programming approach that may be used to predict the 
optimum time for the presentation of a new topic during a CAI segment of 
instruction. Suppes, Loftus and Jerman (1969), by virtue of conducting 
problem solving experiments in a CAI mode, also called for the adaptation 
of curricula to media. The fact that such approaches may tend to be readily 
labelled as esoteric by "practical" educators does not deny the fact that 
their use is sorely needed. 

Three factors contribute to the significance of the present investigation. 
First, a pedagogical problem exists in the junior collage mathematics 
program. Second, the writer has found that for the students involved, 
problem solving in the form of word problems is a difficult and frustrating 
topic. No effort known to this writer has been made to study problem-solving 
difficulty at the junior college level using the structural variables approach. 
Third, greater emphasis is being placed today on computer-assisted instruction 
(CAI) as exemplified by the Coast Community College District where this study 
will be conducted. 
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LIMITATIONS OF THE STUDY 

This study was conducted totally in the computer-assisted instruction 
(CAI) mode. Subjects were presented the problems for solution via IBM 
2741 computer terminals. Responses, as well as computations, were made 
only through the terminalci, thus, subjects' computational skills were 
controlled heavily. By the same token, errors unique to CAI were introduced. 
The most common errors were typographical, and even though they usually did 
not affect the correctness of the solution they did increase student's 
latency of response. 

The subjects included the entire (and only) arithmetic class conducted 
during the Fall semester of 1971 at Golden West College, Huntington Beach, 
California. Students enrolled in this class scored lower than four on the 
SCAT test and were required to take the class in order to achieve a minimum 
proficiency level in arithmetic—compulsory for the Associate of Arts degree. 

This was a singular group, and randomization of subjects within this 
group would, of course, have made for a "tighter" experiment. Unfortunately, 
the small number of students starting the course (62) did not allow for 
this procedure. Hox^ever, randomization of subjects was not as crucial to 
this study as it might be for other experiments. 

The word problems in this study were limited to a level of 
sophistication not greater than ninth grade arithmetic The crucial 
limitations lie here, with the problem set; its make-up, selection, and 
presentation. Ideally, the experimenter should {je able to present problems 
to subjects in such a way that variables increase in a uniform manner. ITiis 
suggests that variables need to be analyzed individually. For example, to 
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test Che ef foots of lln- v.iii.ihl. hU.u ■> ! - nv, r.,.. vi-rb lo noun ratio Lho 
oxpcM-imcMiCar ou«lu t o h..ve ot ,,rohh-n.s I h.l, Ic Hun .-aeadily Incren.so.s 

Lhe ratio along a c nLinunn -.'hilv holding tlu- oUiur sLructnrai variables 
constant— an impossible L.^sk .-.inc.- the vari<,h(es are not. independent:. An 
increase in tl>e verb-noun vaiio torccb an increase in t:ha number of words in 
tliG problem. 

in view of this dilenmia, and to avoid dealing with an enormously large 
set of problems, the 172 problem.s were rando,ai7.ed for presentation to subjects. 
In constructing the pioblems, the writer tried to allov the variables to 
take on diversified value? in their domain of definit:ion. 



CHAPTEF n 



REVIEU OF THb: LITF.l^\fURE 

The set X of variables used in the present study reflects the 
results of these studies by including either the same variables thereof 
proved to be important or modified versions and combination of those 
variables. In this chapter the names of these variables will be capitalized 
and underlined. 

Steffe (1967) explored the effect of using existential quantifiers at 
the beginning of problems, and the use of coiiimon versus different names for 
the elements of sets under coiisideraiion for problen^ solving presented to 
ninety first graders. He found only the latter variable statistically 
significant in affecting the correctness of response. For example, a word 
problem which requires arithmetic operations to be performed with feet and 
inches is considerably more difficult to solve than one which involves only 
inches or only feet. The implication here is that a CO NVERSION variable 
should be included in the set X. 

A superficial, yet logical variable to consider is the length (number 
of words) of a verbal problem. One can assume that longer problems in 
arithmetic are more difficult than short ones. Kilpatrick (1960) verified 
this assumption and found that the length of a problem was more significant 
in predicting difficulty in problem solving than vocabulary or length of 
sentences. Further evidence of the importance of this >;ariable is given in 
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language studies by Braun-Lamesh (1952), Ervin (I96A), nnH Miller and 
Ervin (1963). A natural extension of this variable ^^s used in this study 
is LENGT H as measured by the number of letters in a word problem. Future 
studies might also include the number of syllables as an indication of 
length. To usg all three, or even two of these variables in the same study 
would be redundant since their correlation is very high. 

Burns and Yonally (1964) have sho;>m that, for fifth graders, the sequence, 
or ORDER in which the data appears in a verbal problem is important. 
Problems where the data appeared in the problem in the same order as they 
were needed to achieve the correct solution were found to be significantly 
easier to solve than problems v:here the data did not appear in the needed 
sequence. Rosenthal and Resnick (1971) concurred with and strengthened the 
above results by showing that v;hen data appears in the reverse order as 
needed for solution^ the problem achieves maximum dirficulty. 

The VERBAL CLUE variable, and clue variables in general have had strong 
support from structural variables researchers. Clues in mathematical word 
problems seem to be a favorite topic for research. These studies offer 
diversified results, however, as a result of various types of clues being 
employed in the presentation of word problems for solution. 

Wright (1968) found strong support for verbal clues in arithmetic word 
problems for 382 fifth-grade children. Both correct answers and correct 
process in the pursuit of an answer were heavily affected by the presence 
or absence of verbal clues. Early (1967), as well, found similar results 
with 296 sixth-grade pupils. The .-subjects, when considered as a whole, 
performed better in selecting correct answers and correct procedures in 
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solving the word problems that contained verbal clues; but in some 
subgroups, e.g. high performers in ability tests and these of average 
computational ability, word clues di-i not improve performance. Farly found, 
however, that the most significant differences occurred for children of 
lower ability. 

In related research, Lee (1971) showed that "visually presented cues 
are effective for the acquisition of a rule rather than for transfer." 
Visual clues combined with a weight clue allowed foui-th-graders to acquire 
the concept of the coefficient rule (a F = S) of a linear function 
significantly faster. However, clues did not help on the learning of the 
total linear function (aF + C = S). 

Sherrill (1970), also on vis<ial clues, found that the "effect of 
presenting an accurate pictorial representation as an aid in a testing 
situation was so strong that the low grade average group. . .scored higher 
than any grade average group..." 

These convincing results are not without blemish. For verbal clues 
especially, the fly in the ointment is controversy. V/hat constitutes a 
verbal clue? Arriving at a sound definition of a verbal clue is difficult, 
since there are many exceptions to most definitions. One way around the 
problem is to list the set of terms which are considered verbal clues. 
Jerman (l97l), and Suppes, Loftus , and Jerman (1969), did precisely this and 
the practice has been adopted in this study as well. 

Pursuing the verbal clue variable further leads -to another variable. 
One soon discovers that words used as clues, and even those commonly regarded 
as strong clues, e.g. the words "sum" and "total" can be misleading at times. 
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Consider the problem statement: "Z'he nuin of Georr.fV^ grndon i;: 326. \■n^o.t 
grade must he receive on hir. next lesi. f.o bring hir, tota] to l.on?" Her« the 
words "suKi" and "total" imply addif.ion yel the {.robi-m is solved by subtraction. 

Suydam and Weaver (1S'70) warned researcher of the possibility of 
"grossly misleading" clues. A structural variable that will take up this 
slack is introduced and called the DIgTRACTOR variable. 

The difficulty of mathefiiatics word problems is as much a function of 
linguistic complexity as it is of any single other variable. By its very 
nature, a word problem is a problem in communication. Students who solve a 
problem unsuccessfully often do not know what is wanted. Upon seeing the 
correct solution, typically these students will remark: "Is that what he 
wanted?" 

Has the problem writer failed to communicate, or has the problem solver 
failed to receive properly? The ansx^er to this question must certainly be 
complex— yet it seems logical to assume that a failure in communication is 
most likely to occur if a problem is linguistically complex rather that 
simple . 

Research seems to bear this out. Linville (1969) found that scores of 
UOQ fourth graderd on verbal aritlimetic problems were significantly higher 
on test items with easy vocabulary and/ or easy syntax. He also found that 
high intelligence and high reading achievers scored better that low 
intelligence and low reading achievers. This suggests that the breakdown in 
communication lies on the side of the problem solver; i.e. successful problem 
solvers seem to cut through the "noise" and get to the problem situation 
more easily than poor problem solvers. Assuming, of course, that if the 
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subjects knows what ic wanted he can get the correct Jinswer. 

Further evidence on the importance of reading aoility to problem solving 
in mathematics ws offered by Cottrell (IO67). He found arithmetic 
achievement and reading achievement to have a oorre.lation coefficient of 
.860. Harvin and Gilchrist (1970), as well, found positive relationships 
between problem solving in arifclmietic and reading. They concluded, rather 
interestingly, that arithmetic teachers should teach certain reading skills 
as well as arithmetic skills. 

From the structural variables point of view the researcher would like 
to know what it is about language that makes students stujable when reading 
a word problem. 

The analysis of language and its complexity is the endeavor of the 
fields of Linguistics and Psycholinguistics. These prolific researchers 
have already produced meaningful results. 

From these researchers the DEPTH variable is drawn. Yngve (1960) 
proposed a model for sentence analysis which breaks a sentence down into 
its constituent parts by using a binary rewrite rule. P sentence is thus 
diagrammed as a binary "tree". The starting node represents the entire 
sentence while ending nodes represent each word in the sentence. 

For example, the sentence "The last problem is always too hard" can be 
broken down as follows: 

The l ast probl em^ ^is^ always to o hard 
The last pf'SbTem ig Zlwayrtoo hard 

The" last" problem ig^^ always^too hard 

always too ^ard 




Je last problem is always iToo hard 

(2) (1) (1) (2) (1) (0) 
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The numbers beneath each vord are called "Yngve numbers" and indicate 
the depth of -jmbeddedness of the word In that sentence. This depth is 
measured by the mirabor of nodes (sentence constituents) the reader must keep 
in his short-term memory vhen he is considering that vord. In the exajnple, 
for the word "last", the reader must use his short-term memory to recall 
that "last" preceded (and pertained to) the noun "proble-n" and the verb 
phrase "is always too hard," thus "last" has a depth of two. Conversely, 
upon focusing his attention on the vord "last", the reader (or listener) has 
come to expect (because of our language habits) an noun and a predicate to 
follow. Martin and Roberts (I966), have shOTO using Yngve nunibers, that any 

variations from these unwitten constraints "significantly affects sentence 

retention in the free learning situation." 

As a rule of thumb, the Yngve number of a vord in a sentence can be 

derived easily by counting the number of left branches in the binary tree 

beginning with the uppermost node and ending at the word. 

Yngve applied his model to algebraic sentences quite successfully. 

This success was due to the consistency of mathematical language. When he 

extended it to the English language, however, the model turned out to be 

less than perfect—because of the well known inconsistency of the English 

language . 

With this limitation noted, researcher use the Yngve "metric" for 
measuring the linguistic complexity of the sentences of a word problem as did 
Loftus (1969), who pioneered the use of Yngve numbers in problem solving 
research. Some measure of sentence complexity is bettei than none. To shore 
up the Yngve model with some statistical sandbags the experimenter and 
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Miss Alice Anderton, a graduate student in Linguistics at UCLA independently 
assigned Yngve numbers to the word problems used in the study. The 
correlation coefficient between the two vectors was .76. 

In language studies there is an abundance of interesting experimental 
results as yet untapped by structural variables studies researchers. Consider, 
for example, measuring the "richness" of a sentence (Bruno, 1972) by using 
the ratio of the number of adjectives to the number of nouns. By reversing this 
variable, we get the NOUNS TO ADJECTIVES ratio used in this study. Inherently,a 
sentence with a high nouns to adjectives ratio is rather barren of 
description, and, therefore, harder to understand. Mo studies are offered 
to support or dispute this conjecture because apparently none have been 
conducted in mathematics problem solving. This type of variable should not 
only be used, but extended. Bruno (1972), in analyzing the poetic style of 
German poetry used other linguistic structural varie.bles as well as the 
adjective to noun ratio. Some of these have been adopted in this study. 
Among these are the adverb to noun ratio, NOUl^ TO ADVERB; the number of verbs, 
VERBS; and the number of adverbs, ADVERB . 

The role MEMORY plays in problem solving and learning in general has 
been explored rather thoroughly. Not all results have shown significant 
correlation between memory and problem solving. However, there seems to be 
some relationship between the two— especially b'^tween short-term memory (STM) 
and problem solving (Sieber, Kameya, and Paulson, 1969; IThimbey, 1968, 197O; 
Weir, 1965). Interviews with experienced mathematics teachers have sub- 
stantiated this relationship. These teachers agreed that students able to 
recall a formula by memory generally have an advantage, during problem solving 
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and class discussion, over the ones who cannot recall the formula. 

Returning now to the memory variable, one can include memory as a 
variable in set X being careful to define it in a structural variable sense 
not as a subject-oriented variable. 

Several structural variables studies which analyzed problem solving 
difficulty for problems presented and solved in computer-assisted instruction 
mode (CAI) were conducted at Stanford University by Professor Patrick Suppes 
and associates. 

In Suppes, Hyman, and Jerman (I966), and Suppes, Jerraan and Brian (I968), 
the authors developed and appl ed the theory of multiple linear regression 
to the problem of predicting problem- solving difficulty and latency of response. 
Several analyses were conducted on data trapped by the computer on CAI drill- 
and-practice exercises presented to 270 subjects composed of third, fourth, 
fifth, and sixth graders. The data analyzed were from problems in addition, 
subtraction, and multiplication. 

The structural variables used in the analysis of addition problems were 
MAGSUM (the magnitude of the sum), MAGSMALL (the magnitude of the smallest 
addend), and NSTEPS, itself the sum of the three sub- variables: TRANSFORMATIONS, 
OPERATIONS, and MEMORY. A detailed explanation of the relationships between 
these variables and their definitions is not in place here. Enough 
be said that a specific algorithm, wall explained in the study, was used 
to obtain the values of the subvariables , and thus, the value of NSTEPS. 
The analysis of subtraction problems involved three variables: MAGDIF 
(magnitude of the difference), MAGSUM (magnitude of the subtrahend) and NSTEPS 
(same as in the addition problems). 
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In both analyses NSTEPS was by far the most significant variable for 
predicting percent of errors and success latency. The ranges of r2 for the 

various populations were .16 to .Jk for the proportion of errors and .19 to 
.7^ for success latency. Thus in many cases the model, using only three 

variables, explained problem solving difficuly for addition and subtraction 

problems quite well. 

In two analyses on multiplication problems the variables used were LARGER 
(magnitude of the larger factor), SMALLER (magnitude of the smaller factor), 
and three zero-one variables reflecting the mutually exclusive forms: 

^ ^ = ' a X = c, or X b = c of the multiplication problem (these 

0, 1 variables were used only in the first multiplication analysis). The 
most important variables were found to be, respectively: the zero-one 

variable reflecting the form a x b = , and SMALLER. Consequently, 

multiplication problems of the type a x b = are more difficult (for 

fourth graders at least) than the other two forms; and, given two problems 
of this form, the one with the larger smaller factor will tend to have a 

longer response time. For example, 3x7= , according to Suppes, is 

more difficult than 2x8= , 

In reaching for a finer grained analysis, the experimenters broke up 
the NSTEPS variable into its three component sub-variables: TRANSFORMATIONS, 
OPERATIONS, and MEMORY, and applied the regression equation to 8o fourth-grade 
addition problems. The order of importance of these variables was MEMORY, 
TRANSFORMATIONS, and OPERATIONS. Further refinement brought the MEMORY 
variable together with 0^ (the number of addition operations), and (the 
number of subtraction operations) in the analysis of 19 more problems. Results 



-17- 



shoved the MEMORY variable again wer, more important than the other variables. 
_ These results support the earlier remarks on the MEMORY variable (page ) 
and have led the researcher to. adopt this variable in this study. 

These pioneering studies by Professor Suppes and his associates have had 
a doctrinally, far-reaching effect on subsequent structural variables 
investigations. In addition to the theory, structural variables studies 
concerned with the analysis of mathematics word problems have adopted most of 
the variables. The implications for the present investigation was to adopt a 
modified form of the original, variables. These are: STEPS, MEMORY , NUMBER 
TYPE , and OPERATIONS . 

In presenting the model under discussion at the Conference on Needed 
Research in Mathematics Education, held on the campus of the University of 
Georgia, Suppes (1967), attracted comments from three leading researchers: 
Ralph T. Heimer, Jack E. Forbes, and Joseph M. Scandura. The interested 
reader should consult the report of the conference in order to place the 
3.inear regression model as applied to structural variables studies in its 
proper perspective in the field of research in mathematics education. 

The first structural variables study to analyze data accumulated from 
mathematics word problems presented and solved in computer-assisted 
instruction mode was conducted as Stanford University (Suppes, Loftus, and 
Jerman, 1969). 

Twenty-seven above average fifth graders solved 68 word problems of 
sixth grade difficulty on ten commercially available teletype machines 
connected by private telephone wires to a PDP-1 computer at the Institute for 
Mathematical Studies in the Social Sciences. 
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The intent of the experiment was to extend the previously defined- and 
herein discussed, linear regression model (Suppes, 1966), to the realm of 
word problems in an effort to single out factors that contribute to the 
difficulty of such problems. 

The variables used were of two types: "zero-one" variables and 
"continuous" variables. The first set contained the variables: SEQUENTIAL 
(the problem could or could not be solved by exactly the same steps and 
operations as the problem immediately preceding it), VERBAL CLUE (the problem 
did or did not contain all the verbal clues corrtisponding to the required 
operations), and CONVERSION (the solution did or did not call for the 
conversion of units). The second set contained the variables OPERATIONS (the 
minimum number of different operations* required to achieve the correct 
solution), STEPS (the minimum number of steps required to achieve the correct 
solution), and LENGTH (the number of words in a problem). 
The multiple linear regression equation obtained was: 

Zi = -7.36 + .87Xii + .l8Xi2 + ^02X^3 + 2.13Xii, + .26Xi5 + ^^^^^i6 



* The value of this variable can be misleading. Consider the problem: 

"To have passed his history class, Ralph needed at least 300 points. By how 

many points did he miss passing if his scores were 78, 56, kU^ and 89?" By 

definition, the value of the OPERATIONS variable is 1 (300 - 78 - 56 - liU - 89). 

However, the more "usual" solution requires two operations: 300 - 

(78 + 56 + I4U + 89). Suppes comments on this point but does not state which 

value he would use: 1 or 2 (see page for the resolution of this dilemma 

in this study). 
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with three significant variables: SEQUENTIAL = x, , CONVERSION = 

and OPERATIONS = Xj^, in their respective order of importance. 

The variables STEPS = x^, LENGTH = x and VERBAL CLUE = x did 

5 

not contribute significantly to R^. The model explained about h3% 
of the variance (R = .6?) and had a Chi-squared value of 553.16, 
indicating a rather poor fit. 

The contribution of this study was not its ability to explain 
word problem difficulty, for both R^ and Chi-squared were by far not 
acceptable. Rather, this was the first step toward an organized branch 

of research in mathematics word-problem solving. As shown later, both 
2 

R and Chi-squared will improve in succeeding studies. However, an 
unsatisfactory amount of variance will still be left unexplained. 
Perhaps, as the experimenters themselves admit "...nothing short of a 
full syntactic and semantic analysis will suffice to predict all the 
details that must be accounted for in the behavior of students. "(page ik) 

Loftus (1970) tried to capture at least one facet of the 
syntactic structure of the sentences in word problems-the surface 
structure*— measured by the DEPTH variable (see page ). This 
measure, though admittedly crude was a step in the right direction. 
For it is intuitively clear that the linguistic structure of a sentence 
is important to its understanding; as Professor Suppes conjectured, 
linguistic analysis in mathematics education research is essential 
for the advancement of the science on the word problem solving front. 



A sentence may also be measured by a "deep" structure analysis. In 
this approach the sentence is subjected to defined linguistic 
transformations that will reduce it to a standard format. A measure 
of the number of t ran format ions becomes the index of the depth of the 
sentence. 
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Besides the DEPTH variable, Loftus used the ORDER Vfc:riable 
(see page ). These two, in conjunction with the six variables 
used in the Suppes nl . , brought tho total number to eight: = 
OPERATIONS; x^ = STEPS; X3 = LENGTH; x,^ = DEPTH; x^ = SEQUENTIAL; 
xg = ^/ERBAL CLUE; x^ = ORDER; and x^ = CONVERSION. 

Sixteen sixth-grade students from two "depressed'' area schools 
completed the study (no indication is given on how many began). The 
average sixth-grade I.Q. was 93 in one school and 99 in the other. 
The students solved 100 word problems of appropriate sixth-grade 
difficulty after a practice period of four weeks working with 
computer teletypes (students at one school took longer). The 
regression equation was: 



= -3.21; + .148x^1 + .Olix.g + .88x.^ + .6IX.5 + .20x.g 

+ .13x.^ + .149X.8 



2 

with R = .70 and Chi-squared = 206.7k. Thus, though the amount of 
variance accounted for was respectable, the fit of the model was rather 
poor. Table' 2.1 compares the results of this study with those of 
Suppes et al. 

Note that the robustness of the variables OPERATIONS and SEQUEN- 
TIAL— especially in light of the fact that the subjects in the former 
study wei-e bright and the ones in the latter were average— was quite 
good. The emergence of robust variables is clearly of tantamount 
importance if practitioners of education are to be helped by this 
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TABLE 2.1 



Comparison of the importance* of variables in the studies 
on word-problera solving difficulty by Suppes, Loftus, and 
Jerman (I969), and Loftus (1970), 



Suppes, and others (I969) 


ORDER OF 
IMPORTANCE 


Loftus (1970) 


VARIABLES 


VARIABLES 


SEQUENTIAL 


1 


SEQUENTIAL 


CONVERSION 


2 


OPERATIONS 


OPERATIONS 


3 


DEPTH 


VERBAL CLUE 


U 


LENGTH 


STEPS 


5 


CONVERSION 


LENTGIi 


6 


VERBAL CLUE 




7 


ORDER 


' -1 1 


8 


STEPS 



In the 1969 study importance was seemingly taken to correspond to the 
the magnitude of the regression coefficients. In the I970, the size of 
the partial correlation coefficient was used as an index cf importance. 



type of research- This is not to say that applicability should dictate 

the path of research. 

The reader is cautioned not to allow the apparent robustness 

to lead hin: to false conclusions. These are the results of just tvo studies; 
^ the first did not make clear how the importance of the variables was 

defined, and the second, assuming the partial correlation coefficient is 
: a valid measure of importance, was limited to only l6 subjects. 

Further evidence of the robustness of the variable OPERATIONS, and 
I the appearance of new variables high on the ladder of importance was 

found in a follow-up study by Jerman (l97l). Jerraan re-analyzed the data 

I 

i in the Suppes, Loftus, and Jerman (I969) study with a larger set of 

I predictor variables (21). The following new variables were added (underlined 

variables have been adopted in the present study): 



OPERATIONS 2: The sum of the number of different operations, and 
^ if one of the operations is division, 
2 if one of the operations is multiplication, 
1 if one of the operations is c.ddition, 

ORDER 2: The sum of S^ and one point for each verbal clue 
necessary to establish a new order. 

FORMULA: Has value 1 if knowledge of a formula is required, 
zero otherwise. 

AVERAGE: Has value 1 if the word average appears in the 
problem statement; zero otherwise.* 

ADDITIOW:Has value 1 if the problem requires addition, 
zero otherwise. 

SUBTRACTION: Similar to addition. 

MULTIPLICATION: Similar to addition. 



ERiC I -23- 



DIVISION: Similar to addition. 



S^: Measures the number of displacements of order of operations 
in successive problems, 

S^: Measures the number of displacements between order of 
operations required and that given in the problem 
statement itself. 

RECALL : The sum of: 

(a) formulas needed, 

(b) steps in each formula, 

(c) conversions to be used, 

(d) facts recalled and used from previous problems, 



After some adroit, but artificial manipulation of variables and 
data, Jei-man arrived at the results sumrarized in Table 2.2. We find, 
there, more support for the variable OPERATIONS, and new support for 
LENGTH, DIVISION, S^, and COm^SRSION. 

A second task pursued by Jerman et al., was to compare the 
importance of structural variables for the problems presented in CAI 
versus paper-and-pencil mode. Even though this study does not deal with 
the problem in depth*, the results are very interesting. This analysis 
is displayed in Table 2.3 . There is a definite trend of importance 
for the variables that reflect computational skill, in the paper-and- 
pencil mode of solution. However, the variable LENGTH is also high 
for both modes. In a subsequent, unpublished paper, which Professor 
Jerman was kind enough to send to the author, a different criterion of 
importance was used to rank the variables (amount of variance contribut«d 



I * Jerman candidly warns readers that the comparisons may not be valid. 

Among some questions one may raise are the proper use of the SEQUENTIAL 
, variable. Also, three problems with large Chi-squared values were re- 

I moved from the problem set; and, the problem set of the paper-and-pencil 

group was not the same as that of the CAI group, and for that matter, 

neither were the subjects the same in both groups. 

1 
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TABLE 2.2 



Importance of tvo sets of structural variables using 
stepwise regression analysis on 65 arithmetic problems 
( Jerraan, 1971 ) 



RESULTS WITH 
l6 VARIABLES 


ORDER OF 
APPEARANCE 


RESULTS WITH 
21 VARIABLES 


OPERATIONS 


1 


OPERATIONS 


VERBAL CLUE 


2 


CONVERSIONS 


DIVISION 


3 


LENGTH 


LENGTH 


k 


ORDER 2 


FORMULA 


5 


DIVISION 


^1 


6 


^2 


CONVERSION 


7 


ORDER 


^2 


8 


MEMORY 


Other variables 


9 


DISTRACTORS 


Other variables 


10 


Other variables 









TABLE 2.3 



Comparison of the importance of structural variables in 
predicting word-problem solving difficulty for problems 
solved in CAI versus paper-and-pencil mode usin^ 65 CAI 
problems, 30 paper-an£-pencil problemft ,'-ana 19 'va^^iables 



CAI MODE 


ORDER OF 


PAPER-AND 




IMPORTANCE 


PENCIL MODE 


OPERATIONS 2 


1 


LENTGH 


LENTGH 


2 


N0MC2* 


ORDER 2 


3 


QUOTIENT* 


RECALL 


k 


DISTRACTOR* 


^2 


5 


C0LC2* 


.56 


r2 


.87 



* These are computational variables introduced especially because 
these subjects had to perform their own computations. Briefly, N0MC2 
measures regroupings in multiplication?,, QUOTIENT measures the number 
of digits in division, C0LC2 measures viie number of columns and 
regroupings in addition and subtraction problems. 
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to the total variance). For this criterion of importance, the CAI variables 
were ranked, respectively: OPERATIONS 2, CONVERSIONS, LENGTH, ORDER 2, and 
DIVISION. Whereby the paper-and-pencil variables were ranked, respectively: 
MULTIPLICATION (N0MC2), DIVISION (QUOTIENT), LENGTH, and DISTRACTORS. 
These new results strenghten the above conclusions regarding the computational 
variables . 

The large amount of variance accounted for (.87) in the Jerraan study 
for paper-and pencil mode must be looked at in the light of the ratio of 
variables to problems. Statisticians warn that if this ratio is allowed 
to approach one, then the dimension of the rank space approaches the number 
of variables, and the regression approaches a urique solution with R =1. 
Indeed, one may solve a set of simultaneous equations in this case, where 
the sum of squres is zero. The ratio for the paper-and-pencil mode was 1.6, 
and that of the CAI mode was 3.U2. This, more than the effect of the 
variables may be the reason for the high R^ value in the paper-and-pencil mode. 
Along these same lines, the study by Loftus(l970) , with a ratio of 12.5 is 
least open to this type of criticism—including the present study, which 
has a ratio of 5.7. 

This chapter closes -with a look to future research on structural 
variables studies. More of these studies will be emanating from Professor 
Suppes at Stanford University, and Professor Jerman, now at the Pennsylvania 
State University. Suppes is presently running a more comprehensive study, 
while Jerman has revised his 1971 study in an unpublished report by 
Jerman and Rees. Ongoing studies at Penn State include: " Linguistic 
Variables in Verbal Arithmetic Problems," by Ed Beardslee; "Arithmetic, 
Linguistic , and Algebraic Structural Variables That Contribute to Problem 
Solving Difficulty of Word Problems in Algebra," by Blair Cook; "Predicting 
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the Relative Difficulty of Problem-Solving Exercises in Arithmetic-," by 
Max Jennan; and "Structural Variables in Problem Solving Exercises Solved 
by Prospective Elementary School Teachers," by Max Jennan and Sanford Miram. 

All these studies, as well as the ones reviewed in this paper, approach 
problem solving difficulty by examining the structure of the word problem. 
All basically assume that structural variables have the ability to pre- 
dict difficulty and latency of response. There is no reason to doubt, at 
this time, taht this assumption is incorrect; for amidst the variety of 
variables and approaches there has been a visible thread of consistency. 
Much work needs to be done, however, to wean this young branch of research 
into maturity. 
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CHAPTER III 



DEFINITION OF VARIABLES AND THEORETICAL CONSTRUCTION 

This section is used to define the structural variables and the 
dependent variables and develop the notation needed for applying three 
statistical models to the analysis of the data collected. We begin with 
the dependent variables. 

Part of the APL/360 software package written exclusively for this 
study is described in AppendixZ. The functions in this software package 
were designed to present the problems to each subject and trap and save 
response data. The following dependent variables were chosen from the 
response data as being representative of problem solving difficulty: 

Pi = PERCENT CORRECT: (the only one used in this paper), measures the 

percent of correct responses for problem i. 
1^ = LATENCY : measures the average number of seconds used by the 
experimental group to arrive at an answer for problem i. 
= SUCCESS LATENCY: measures the average number of seconds used 
by that subset of the experimental group that arrived at the 
correct answer for problem i. 
= STEPS: measures the average number of steps used by the experi- 
mental group to arrive at a solution for problem i. 

Consideration is now given to the task of defining the structural 
variables — the independent variables of this study. Thirty variables 
were selected on the basis of proved importance in previous studies and 
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conjectured importance based on surveys with junior college raathema- 
tics students and teachers- Before proceeding, however, it should be 
mentioned that the variables SUPERFLUOUS DATA and MISSING DATA were 
not discussed in the review of the literature. But they are included 
as variables in this study. The author apologizes for this oversight 
in the present paper, and assures the reader that in the dissertation 
in progress, of which this paper is only a part, these variables will 
be given their proper due. One can intuitively see, however, that these 
variables, especially MISSING DATA, play an important part in the ^problem 
solving process. Other variables not specifically mentioned in the 
review of literature chapter are derivatives of reviewd variables. 

The set X of structural variables contained the following elements: 
^1 = OPERATIONS; The minimum number of different operations required 
to achieve the correct solution, plus: 

1 if at least one addition is required, 

2 if at least one subtraction is required, 

3 if at least one multiplication is required, and 

4 if at least one division is required. 
Range of = (2, 3, 4, 14) 

X^ = DIVISION: The minimum number of times division needs to be used 

in order to achieve the correct solution. 
Range of X;^ = (0, 1, 2, 3, n) 
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= HIERARCHY; has value 16,8,4, or 2, respectively as the first 
operation required to achieve the correct solution is division, 
multiplication, subtraction, or addition. 
Range of = (2,4,8,16) 

STEPS; the minimum number of steps (or binary operations) 
required to achieve the correct solution. 
Range of - ( 1, 2, 3, ... , n) 

= LENGTH; the total number of letters and digits in the problem. 
Range of = (1, 2, 3, ... , n) 

Xg = DEPTH 1; the sum of the Yngve numbers of all words in the problem. 
Range of Xg = (1, 2, 3, ... , n) 

= DEPTH 2: the value of x^ divided by the. total number of words 
in the problem. 
Range of xy = ( rational numbers between 0 and 4) 

Xg = VERBS ; the number of verbs in the problem. 
Range of xs = (1, 2, 3, ... , n) 

Xg = ADJECTIVES; the number of adjectives in the problem. 
Range of xg = (0, 1, 2, 3, ... , n) 
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X^Q - NOUNS: The number of nouns in the problem. 
Range of Xio= (1,2,3, n) 

X^^ = ADVERBS : The number of adverbs and adverbial clauses 

in the problem statement. 
Range of X^^ - (0,1,2, n) 

Xi2 = PROXOUNS : The number of pronouns in the problem 

statement. 
Range of Xi2 = (0,1,2, n) 

^13 " NOUN TO VERB RATIO : Measures "inverse assertions" 

by the ratio Xio/Xa = number of nouns/number of verbs. 
Range of X^s = (positive rational numbers less than 5) 

^14 = NOUN TO ADJECTIVE RATIO : Measures the inverse of the 
"richness" of the problem statement by the ratio Xio/Xp 
number of nouns/number of adjectives. 

Range of X14 = (positive rational numbers less than 10) 
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= PRONOUN TO NOU N RATIO: Measures the amount of 

"indirectness" in the problem statement by the ratio 
Xi2/X|o= number of pronouns/number of nouns. 
Range of = (positive rational numbers less than 10) 



^16 = VERB TO ADVERB RATIO: Measures the inverse of "modification" 
in the problem statement by the ratio Xg/Xn = number of 
verbs /number of adverbs. 

Range of X25 = (positive rational numbers less than 10) 



Xi7 = SEqUENTIAL; Has value one if the problem does not follow 

a problem that is mathematically equivalent to it, i.e., the 

problem cannot be solved in exactly the same manner as 
the preceding problem. The value of X^y is zero other- 
wise. 

Range of X17 = (0,1) 



X18 = ORDER 1 : Has value zero if the numbers in the statement 
of the problem appear in exactly the same order as they 
are needed for solving the problem, i.e., if they appear in the 
order customarily prescribed by X3; one otherwise. 

Range of X^g, = (0,1) 
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= ORDER 2 ; The minimum number of permutations required 
to change the sequence of the numbers in the statement 
of the problem to the sequence customarily required 
by X3. A number missing from the statement of the 
problem is inserted into the solution sequence increasing 
Xig by one* 
Range of X^g = (0,1,2, n) 

X20 = VERBAL CLUE 1 ; Has value zero if the problem has at 

least one word that is customarily associated with one 
of the four basic arithnifitic operations and that 
operation is required in orc!er to solve the problem: 
otherwise, the value of X20 is one. Since this variable 
is questionable well defined, we include in Appendix 
the words considered verbal clues in this study. 

Range of X20 = ^0,1) 

X21 = VERBAL CLUE 2 : Measures the number of verbal clues 

missing from the problem statement. 
Range of = (0,1,2, n) 

X22 " DISTRACTOR : Measures the number of words in the 

problem — usually considered verbal clues ~ that 

are misleading clues. Thus, if the word "sum" appears in 
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the problem statement and ii does not refer to an 
addition operation then X22 is increased by one. 
Range of X22 = (0,1,2, .. ., n) 



X23 = SUPERFLUOUS DATA: Has value one or zero, respectively 
as the problem statement does or does not contain data 
irrelevant to the solution of the problc 

Range of X23 = (0,1) 



Lem, 



erJc 



X2/i = MISSING DATA: Has value one if the problem solver 

himself needs to enter data necessary to the solution 
of the problem. 

Range of X24 = (0,1) 



X25 = REUSED DATA : Has value one if any numbers in the problem 
statement must be used more than once in arriving at the 
solution; zero otherwise. 

Range of X25 = (0,1) 



X26 = NUMBER TYPE 1: Has value one if more than 25% of the 
numbers involved in the solution of the problem 
(including the answer) are less than one or greater than 
10,000. X2g is zero otherwise. 

Range of X26 = (0,1) 
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= I ^UMBHK TYIMC 2 : llns value one If iho correct answer \^ 
not a wiiolc numbCM- or a clocimal exact: to the hundr^'lh 
position (i.e., Lf liio answer is in an "unusual" rf)rni 
such ns ()8.258). X>> ^ is ^oro oriierwise. 
Range of X;^7 (0,1) 

^28 ^ CONVERSION : Has value one if conversion of units from 
one denomination to anotiier is required to achieve the 
solution; if a percent iias to be "changed" to a decimal 

(or vice versa). zero otherwise. 

Range of X28 = (OJ.) 



X29 = MF.MORY I : is the sum of: 

(a) X], and 

(b) the number of numerals in tiie problem and 

(c) the number of formulas needed 

(d) tiie number of conversions needed, and 

(i) 1 if the conversions involve squared units 
(ii) 2 Lf the conversions involve cubic units. 
Range of X29 = (1,2,3, n) 



X3Q = MEMORY 2 : Ts the sum of the number of 
i-ri9^" (a) formulas needed, and 



(h) si:ops in cnc.U roninil:i, mu! 
(c) convcM's lows . 
Range of X3Q = (0,1,2, . . . , n) 



The thirty variables just defined make up a rather formid- 
able set — certainly too large to be of practical value. Ideally 
five or less variables should be used, and optimally, two or three 

One can readily see, hov;ever, that, problem solving will 
not be pinned down to onlv a handful of variables. The imoosing 
amount of research on problem solving has spawned many seeminglv 
different variables that contribute to problem solving difficulty. 
Uliether all these variables are indeed different or can be 
reorganized into a smaller subset is yet to be shouTi. The writer 
doubts that even this smaller subset can be less than five in 
cardinality . 

Yet, proliferation of variables is not a disturbing 
problem in structural variables studies using the regression 
model. One can easily "skim off" a workable number of variables 
from the regression equation. For this, one does need some 
criterion of ^'importance". The articles by Darlington (1968), 
and Linn and Merts (1969) should providi^ excellent guidelines. 
This study will show that one can choose the first three variables 
entering the regression equation and come away with more than 
60 percent of the variance explained* 

The notation needed for the statistical models used in 
this study will nou be developed. The primary model is Linear 



l\oj>ros.s i on. Disrrimin.ini An.ilvsis and C'.niuii i r.i 1 Ci<u"ro Ion 
comprise tho secondnry modol«. 

Lot x^^ bo the value of t:he J^^^ strucCural variable for 
che problem; Xj^^j is an element of Che set X of Chapter 2: 

X = (xl, X2, . . . , Xkj , 
where each element now takes on a second subscript, i, Cor 
i = 1,2, n, and n is the total number of problems. We 

rewrite X as : 

^i = (^jl>'V> •••>^ik)> ^=i>'-^> ---n. 
The sec A of Chapter 2: 

A = Ui, a2, . . . , aj^-} 
remaines unchanged- .^n element aj of set A is a statistical 
parameter derived from linear regression analysis and represents 
the x^eight given to variable x j . 

The stepwise linear regression model (BMD02R) will be used 
to derive the set A for the set X of structural variables, and a 
constant a^, such that a weighed linear combination Pj^^, defined 
by the equation 

Pi "^-^j^ii ^o> ^ ...,n 
j 

will predict the dependent variable p-^. 

The reader should note that even though the observed 
data will always yield a p^ of value greater than or equal to 
zero but less than or equal to one, the predicted value need 
not be restricted in such a manner as Figurejl shows in the 
case of one independent variable. 
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Pi(x) 



Values of x for 
which 'pj[^(x)>l 



FIGURE 3.1 

To avoid this statistical quirk a logarithmic trans- 
formation is used. This will allow the predicted values, p^^, 
to stay within the range of values of a probability function, 
0£^j^il. The transformation* suggested by Suppes, Jerman and 
Brian (1968) is used: 

= log pi y 0 <1 



The graphical effect of this transformation on the pi's 
can be seen in Figure 2 



;:\. 



I 



V 



Pi 



FIGURE 2 



*Other transformations may be used. For example, Zj|^=Cot '/r 
will achieve similar results. 
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For cases where pi = 0 or pi = 1 (Zj^ = lOO) we adulterate the 
transformation by using pi = ,001 and pi = ,999 respectively. 
Also, = 3 for 0£pi< ,001, and = -3 for l>pi> ,999. 
The regression analysis is then performed on Z. : 

" ^ ^ij ^'^o 
j 
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CHAPTER IV 



DESIGN 

The objective in the introductory chapter is here 
f onnalized; 

OBJECTIVE; Use stepwise linear regression (BMD02R) and 

1 - pi 

the logarithmic transformation Zi = log pi to obtain 
a set of weights, A= [aQ, a^, a^o] that will yield a 

regression equation: 



30 

Zi = E a-x^. + an , 1£ i 1172 
j=l ^ 



Identify: 

a. the order of importance for the elements of set X, 

and thus obtain the ordered set X-^= (xj >-X2 > - . . >X3^) . 

b, the F-value which measures the goodness of the fit 
of the model. 

SUBJECTS ; Sixty-five students enrolled during the Fall semester 
of 1971 in the arithmetic course known as Math 005 at Golden West 
College, Huntington Beach, California, comprised the initial set 
of subjects. Because of normal academic attrition, 44 students 
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completed Che course. And, since the study coincided with the 
course, students compleLod the srndy. No response data of 
withdrawn students wa.s u.<;ed In the study. 

The choice of experiment location was predicated primarily 
by the requirement of finding a junior college in Southern 
California with computer facilities capable of handling the 
study. Golden West College met this requirement. 

Students take Math 005 at Golden West College because 
they wich to attain the Associate of Arts degree, but have 
scored less than four on the mathematics part of the SCAT.* 
A grade of C or better in the course rectifies this low score 
and fulfills the imposed minimum proficiency requirement in 
mathematics set by the college. 

Generalizations based on the results of this study 
should be made keeping in mind that the subjects in this 
study were, insofar as mathematics was concerned, slow learners. 
Lest one be tempted to lalel all these students as slow learners 
in general, the experimenter can attest from experience that some 
excell^^.d in other fields. 

Golden West College draws its students primarily from 
the cities of Huntington Beach, Fountain Valley, Westminster, 
Garden Grove, and Seal Beach, California. The combined 
population o.f these cities is approximately 400,000. All cities 
are in the county of Orange, California, and comprise part of 



*See the footnote on page of Chapter I . 
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what is facetiously called the "bedroom community" of I.os 
Angeles. The socio-economic background ranges from lower to 
upper middle class. 

HARDWARE. The problem in this study were presented to, 
and solved by students via IBM 27A1 computer terminals located 
in the Multi-Media Center at r7olden West College, Huntington 
Beach, California. Thirty-four terminals were available to 
students daily from 8 a.m. to 11 p.m. The terminals were hard- 
wired to an IBM 360/50 computer, located ten miles away at 
Orange Coast College, Costa Mesa, California. The system 
operated in OS mode, had 20 disk paks, three of which were used 
exclusively for CAI. 

SOFTWARE . APL/360 was the language used to code the programs 
and analyse the data. It is a new, interactive language which 
is highly mathematically oriented, yet readily adaptable to 
computer-assisted instruction programs — a spin-off not 
anticipated by the creator of the language. Dr. Kenneth Iverson 
(1967), presently with IBM. The functions were coded by Mr. 
John R. Clark, CAI consultant to the Coast Community College 
District under the direction of the experimenter. Some of these 
programs are listed in Appendix 2 . 

The presentation format used by this software package imi- 
tates closely that of the Stanford studies cited previously. But, 
the packages were coded totally independently. Imitation was by 
necessity rather than choice, since comparative investigations 
were planned as part of the present study. 

Some changes in the software have been initiated by the 
experimenter, such as the addition of hints, in an effort to con- 



duct more detailed, future studies in this area. 

PROCEDURE . During the first meeting of the semester, 
students were oriented to the course by the regular instructor. 
He explained that part of the course requirement was the completion 
of twenty word problems per week to be presented and solved via com- 
puter (time factors were adjusted in order to make this unusual require- 
ment fair to the students). 

The experimenter introduced the entire group to general pro- 
cedures and a video tape was used for illustration. Tn subsequent 
meetings the class was grouped, and each group was given a live 
demonstration on a computer terminal at the Multi-Media Center. 
Each student was assigned an individual computer number (with lock- 
word) and instructed to use only that number for the entire semester. 
Each student chose a time slot of one hour per week convenient 
to him. This guaranteed the student a computer terminal for that 
hour. 

The logic of carrying out the computer problem 
solving was itself taught through a pre-planned computer program 
as well as through demonstration and question and answer periods. 
Students used the first two months of the semester — and, a toial 
of 120 problems (not used in the analysis) to practice solving 
word problems on the computer in the context intended by the 
experiment . 

The following examples, illustrate the sequence of steps 
one particular student used to solve a problem. 

The student *s responses are always preceded by the symbol 
"q", and have been underlined to facilitate reading. 
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1 

BILL nn.TOlPS CLASS I'ROGMM AT CYPRESS COLLECF, LAST SF.MFSTFE rONSrSTEP 
OF BIOLOGICAL SCIKNCK, 3 UNITS; CALCULUS ?, 4 UNITS; ENGLISH \P ^ 
UNITS; PHYSICAL ED, 1 UNIT, AND U,S, HISTORY, 3 UNITS. HOW MANY UNITS 
IS HE TAKING THIS SEMESTER IF HE IS TAXING 3 MORE THAN LAST SEMESTER? 

ylf- 3 
C^ 1 

14 
£■•<- 17 

YOU GOT THAT ONE 



G A+3+A+C+A 
□ D+Z 
C E* 

FIRST RATE, 



2 

AN AUTOMOBILE SALESMAN WILL A-LLOW MR. ANDERSON A TRADE IN ALLOWANCE 
OF ^1855 TOWARD THE PURCHASE OF A NEW GTO PRICED A'^ $'374'5 
miAT IS THE DIFFERENCE MR. ANDERSON HAS TO PAY FOR THE NEli CAR? 

A-*- 1865 
5f- 3749 

C<- 18 '.'4 

YOU GOT THAT ONE 



□ B-A 

a c* 

VERY GOOD , 



[ 27/0 FLOCKS OF SHEEP WERE COUNTED. 'THE FIRST CONTAINED 40 SHEEP. 

THE SECOND HAD 10 MORE THAN THE FIRST. WHAT WAS THE TOTAL 
NUmUR OF SHEEP? 



A<- 40 
B-k- 10 

D /i+iEf 

C<- 50 

a c 

LK- 50 

□ D* 

NEGAl'IVE, YOU MISSED THIS ONE, 



* The symbol*- is read . . . is assigned the value ..." 



Th<i chronoloKV of tlicso problems l.s ;is follows: i ho compuior 
typed out everything up to the first quad, incJudinp the numrrnlrt 
whLch appear under the probaem. The computer then paused, RivLng 
control back to the student and was ready to execute any "legal" 
command given it by the student. For instance, in the first problem, 
the student commanded the computer to perform the addition A+B+A+C+A. 
The computer responded with D ^U. The remaining steps are self 
explanatory. 

Note that the computer does not respond to an answer, right 
or x^rong, unless the student commands it to recognize an answer by 
the preassigned method of typing a sing.i.e letter followed by the 
asteriks (*) . Thus, a student has the perogative to take as many 
steps as he wishes, as much time as he wishes. 

The Problems 

The 172 fxithmetic word problems used in this study were 
selected as follows: 100 were identical to the problems used in 
the Loftus (1970) study. Minor changes in four problems had to 
■be made. Since percent problems prove to be exceptionally diffi- 
cult for Math 005 students, 30 problems on percent, similar to 
problems these students are i-equired to solve in the course, were 
constructed and included in the study. The remaining h2 problems 
vere of a general nature, covering concepts a person proficient 
in arithmetic should know, yet, these problems were still of the 
Math 005 caliber. Among the last k2 were 20 problems derived from 
problems suggested in the CUPM (l97l) publication pertaining to 
•this type of course. 
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CHAPTER V 



RESULTS 

This chapter presents the results of the experiment. To facilitate 

reading, the objective is repeated here, and then followed by the pertinent 

results. Before proceeding, the definition of variable importance, mentioned 

in Chapter IV, must be formalized: 

Definition of importance: Let x. and x^ be tv^o variables in 
the regression equation. 

y = y a-x. + a , 

Also, let and R^ be the amount of variance accounted for by the 
respective regression equations: 

y = ^ ^i^i 
iii 

y = I CiX. + Cq. 
i 

Then x. will be considered more important than x^, written 

xj > xJ, if < R^. 

This shall be the primary criterion of the importance for the variables 
present in the linear regression equations in this study. However, to 
test the "robustness" of the variables, other criteria of importance will 
be used in the sequel. 

Objective . Use stepwise regression and the logarithmic transformation 

2^ = log (l - Pi)/p^ to obtain a set of weights, A = {ag, a-^, a^gJ 

that will yield a linear regression equation: 

^i " ^ ^i^i 1 1 i 1 172 

i 
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Identify: 

a. the order of importance of the elements of set X using the 
definition stated above; thus, obtain the ordered set 

X' = {xf > ••• >x3o}. 

b. the significance of the model by the F-value in the analysis 
of variance for the regression; and the Chi-square value vhich 
measures the goodness of the fit. 

The characteri.-tics of the regression model are presented in Tables 
1, 2, 3, and I|, as well as Figure 1. The coefficients of the regression 
equation: the elements of set A, are listed in the first column of Table 1. 
Also in Table 1 are the standard errors, computed t-values, and the amount 
of variance lost by the removal of each of the variables in turn from the 
model. Six variables had significant t-values, but the most impressive 
was that of the MEMORY 2 variable. This variable also had the largest drop 
in the R-squared when it was removed from the model. These two criteria 
have Obvious theoretical ties, however, and it is not surprising that the 
variable scores highly in both cases. Table 2 shows the order of appearance 
of the variables into the regression, the multiple R, R-squared, and the 
increase in R-squared at each step. The variables MEMORY 2 and ORDER 1 
alone account for over 50% of the variance in problem solving difficulty. 
It will be seen, however, that the first ten variables listed in Table 2 
are not In the same order as given by the definition of importance adhered 
to in this study. The reason for this discrepancy is that the order of 
entry into the regression equation does not necessarily reflect the ti-ue 
contribution a variable makes to the total variance explained. Nevertheless, 
with a relatively small set of variables, 70% of the variance has been 
accounted for in problem solving difficulty for the junior college group 
sampled. Table 3 displays the analysis of variance for the regression. 
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TABLE 1 



REGRESSION COEFFICIENTS, .'iTANDARD ERRORS, COm'TED T-VALUFS, 
AND R-SQUAREU DROP CAUSED BY THE RFh'OVAL OF EACH VARIABLE IN 
TURN FOR OBJECTIVE 3 (172 PROBLEMS, AND 30 VARIABLES.) 



VARIABLE 


REGRESSION 


STANDARD 


COMPUTED 


RSQ 










m r/A T TIT? 


PHQr 


0 


CONSTANT 


■"2.6253 








1 


OVER/' TIONS 


0.0354 


0.0331 


1.0711 


0.0021 


0 


DTVTSTON 


Ci 111 f> 


yj . iU / D 


1 . U vl 1 / 






HIERARC^^Y 




. U J. J. D 




U . UUCl 


l\ 




V . V X O 4^ 






U .UUU J 


5 


LENGTH 


\J . V V •! V 






U .UU OX 


5 




"0.0060 


0.0023 


-2.61.47 ** 


0.0124 


7 


DEPTH 2 


n 1 nuu 


U . J. lie 


r\ O O Oil 

U . y oo4 


U .001b 


Q 


r UIXUU 






A H It O C 


0 . 00 z*T 


Q 




0 nnRu 


Ti C\ oil i 


u . ^i^N^y 


0 .0001 


10 


NGUNS 


V . U 0 J.D 


C\ O O '7 


~z . /uyo 


0 .0133 






0.1C73 


0.0616 


1.7427 * 


0.0055 




PPOh^niJMc; 


U . UzUO 


r\ A tr ii o 


-0.3760 


0.0003 






U . Uolz 


U . U^?oO 


•1 o o o o 

1 .3882 


0 .0035 




IVUUI\0 •HUU r*\^X J. V uO 


v . uoyu 


U .Uoz / 


1.1932 


0.0026 


ID 




A "7 A c c 

U . / U 0 D 


r\ C "7 Q o 


1.0380 


0 .0020 


16 


VERBS '.ADVERBS 


0.0361 


0.0294 


1.2275 


0.0027 


17 


SEQUENTIAL 


0.0344 


0.0945 


0.3637 


0.0003 


18 


ORDER 1 


0.3396 


0.1194 


2.8428 * 


0.0147 


19 


ORDER 2 


0.0415 


0.0696 


0.5962 


0.0007 


20 


VERBAL CLUE 1 


0.1156 


0.1257 


0.9195 


0.0016 


21 


VERBAL CLUE 2 


0.0336 


0.0540 


0.6214 


0.0007 


22 


DISTRACTORS 


0.0721 


0.0711 


1.0149 


0.0019 


23 


SUPERFLUOUS DATA 


0.1591 


0.1302 


1.2218 


0.0027 


2k 


MISSING DATA 


0.1945 


0.1727 


1.1262 


0.0023 


25 


REUSED DATA 


0.0296 


0.1483 


0.1996 


0.0001 


26 


NUMBER TYPE 1 


0.1180 


0.1183 


0.9975 


0.0018 


27 


NUMBER TYPE 2 


0.2200 


0.1406 


1.5654 


0.0045 


28 


CONVERSION 


-0.1784 


0.1684 


-1.0590 


0.0021 


29 


MEMORY 1 


0.0741 


0.0537 


1.3806 


0.0035 


30 


MEMORY 2 


0.2876 


0.0425 


6.7595 *** 


0.0829 



* P<.1 P<.01 *** P<.001 



TAELI 2 



SE^mrmi apfeapaiice of we varubl---'-^ ivto 

nEGPE3SI0, EQUATION FOR OBJECl'I\rE 1. ALSO i^;i6wK 
ARE THE WLTIPLE R AND -SQUARED, 171^ INCREASE 
IN n-SQUARt^ (17? PROBLEl-3, AND 30 VA'{IABl.7S) 



m,TIPLE n INCRFA 

^ SQUARED IN RSC 



30 !'E!''JRY 2 

18 ORDER 1 

21 VERBAL CLUE 2 

1 OPERATIONS 

15 PRONOUNS ::-10UNS 


0.6398 
0.7559 
0.7898 
0.8023 
0.8115 


0.47-;8 
0.57J4 
0.6:^38 
0.6437 
0.6587 


0.475? 
0 .0^56 
0.0524 
0.0199 
0.0150 


22 DISTRACTORS 
11 ADVERBS 

6 DEPTH 1 

1.6 VERBS -.ADVERBS 

23 SUPERFLUOUS DATA 


0.8177 
0.8226 
0.8281 
0.8351 
0.8397 


0.6686 
0.6767 
0.6857 
O.G':'7'4 
0.7051 


0.0D99 

V . \/ \/ ^} v 

0.0091 
0,0115 
0.0077 


-27 NUf-JBER TYPE 2 
I'i NCUNS -.ADJECTIVES 
2 DIVISION 
26 .V?;/liB£7? ri?-'? 1 


0.841(1 
0.8456 
0.8468 
0.8476 
0.8485 


0.7125 

f] 7 inn 

0,7171 
0.71Qn 
0.7200 


0.0074 
C .0025 
0.0020 
0.001^ 
0.0015 


10 A'ni'K- 

13 NOUNS -.VERBS 

29 ;/ZiW7tr 1 

0 VEF.BS 

2H MISSING DATA 


0.3541 
0.8554 
0.8564 
0.8576 
0.8583 


0.729r: 
0.7317 
0.733if 
0.7355 
0.7375 


0.0095 
0.0022 
0.0017 
0.0021 

0.00?:] 


38 CONVimSION 
20 ICT£?/.i C7iy/? 1 
7 DSPSff 2 
19 O^Dffi 2 
12 PRONOUNS 


0.8599 
0.8609 
0.6017 
0.8622 
0.8623 


0.739i; 
0.7i;li 
0.7i^25 
0.7U34 
0.7436 


0.C019 
C.0017 
0.0014 
0.0009 
0.0002 


17 SEQUENTIAL 
25 i?^^^^!) 
3 HIERARCHY 
V ADJECTIVES 
U STEPS 


0.8625 
0.8625 
0.8626 
0.8626 
0.G527 


0.7U39 
0.7441 
0.7442 
0.7i;43 
0.7443 


0.0003 
n.O002 
0.0001 
O.OOOi 
0.0000 
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The F-value of 13.673 vas significant at the .0001 level (see part b of 
Objective l). 



TABLE 3 

ANALYSIS OF VARIANCE FOR THE REGRESSION' 
ANALYSIS FOR OBJECTIVE 1 



SOURCE 


D.F. 


SUM OF SQUARES 


MEAN SQUARES 


F-RATIO 


Regression 


30 


IIU.605 


3.820 


13.673* 


Residual 


Ikl 


39.396 


0.279 




Total 


171 


15'*.001 







*P<.001 



The goodness of the fit of predicted versus observed p 's was 

1 

calculated by means of the Chi-squared formula: 

= Z (fi - PiN)/(l - Pi)piN, 
where f- = observed frequency of correct responses, p. = the predicted 
percent of correct responses, and N = the number of students. The value 
of Chi-square was 1382.7, indicating a rather poor fit. However, a closer 
analysis showed that six problems had exceptionally high Chi-square values. 

Figure 1 shows the graph of the predicted versus observed percent 
of correct responses (p^ and p-). The graph was ranked according to the 
observed difficulty, thus the easiest observed problem is closest to the 
origin, while the most difficult problem is at the farthest point on the 
horizontal axis. Perusal of the graphs in Figure 1 will show that the 
model gives a much better account of the easier problems than the hard 
ones. Perhaps by choosing a more heterogeneous set of problems this 
difficulty can be ironed out. Nevertheless, the trends are the same. 
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Table k lists the variables according to four criteria of importance: 
the first, by the definition used in this study; the second, by order of 
introduction into the regression equation; the third, by size of the decoded 
beta weights; and the fourth, by the size of the partial correlation coeffi- 
cient. Although in this study, the importance of a variable was defined 
according to the first criterion, that is, by the amount of R-squared drop 
caused by the removal of that variable from the regression model. Table h 
offers a measure of "robustness" of the variables, within the confines of 
this study. Note that the variable MEI40RY 2, consistently appears first 
on all orderings of ijnportance. The significance of this fact is under- 
scored by the strong showing this variable made in Tables 1 and 2. The 
ordered set X" (see part a of Objective l), is shown in column one of 
Table U. The ordered set X" can be achieved by reading the column from 
top to bottom. 
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TABLF t+ 



omim OF mwE'PANCE of the 30 STPUcTimr, vaftablfs as 

MEA.WRED FY FOUR CRITERIA OF IWORTAKCF . TRF FIRST 0^ 
THESE IS THE CRITERJOK OF IHPORTAHCF AS h'!l';ASUPFn r^Y 
THE dEFIllITIOP, THE SECOND IS THE ORDER OF ARVEm.m^ 
OF THE VARIABLES IIITO THi^ RFGPFSSIOV ^OVATWIK T"r 
THIRD y 3Y THE DECODED BETA WICWS , AND Tlh: FOUR'FF 
BI THE SIZE OF THE PARTIAL CORRFT-ATION COFFFICTENT 
(VAFIAFLES HAVE BEEF ABBRfVIA7W) 



DEFINITIOn OF ORDER CF DECODED BETA PA^TTAL CORE, 

OF mmiAECE AFPEAPAm: WIGHTS COEFFICIENT 



MimORY 2 


MEMORY 2 


I'fr'.'-fCRY 2 


fiPWRY ? 


OF.UER 1 


ORDER 1 


NOUNS 


ORD"^ 1 


Nomis 


n-RBAI CL2 


LENGTH 


W.RBAL CL2 


DEPTH 1 


OPF^ATTOfT^ 


Uilrl r 1 


f )yrnA ^T.O N ^ 


LENGTH 


PRON-.NOVN 


0RD¥:R 1 


PRCN'.mUN 




DISTRACTOR 


mNORY 1 


Vm;'.ADVRB 


NOWR TYP2 


AVVtJRBF 


ADVFISS 


NOUNS 


WU:\xVERB 


DEPTH 1 


VERBS 


DISTRACWP 


MiXCR.l 1 


i'';7?r lADlW 


NOVNiW.RB 


DEPTP \ 


SUP'S DATA 


SUP'S DATA 


NOUN'.ADJTS 


Nl'M'R TyP2 


VERF lADVRB 


NUhfR TyP2 


OPERATIONS 


SUP'S DA.TA 


NOUHiADJTS 


nOUNiADJTS 


PRON'.NOUN 


AFWP.FS 


VERBS 


DIVISIOK 


i^JRB :ADVRB 


NOmiiADJTS 


MIS'C DATA 


NUM'R TYPl 


NUM'R TYP2 


NOUN: VERB 


OPERATIONS 


LENGTH 


MXS'G DATA 


VERBS 


CONVERSION 


NOUNS 


COm.^.SION 


yiF'G DATA 


PROU-.NOUN 


NOUN:\rERB 


SUP'S DATA- 


DIVISION 


DIVISION ' 


1-T.MORY 1 


DlVISIOy' 


CONW.PSION 


DISTRACTOR 


VERBS 


NUM'R TYP\ 


VT^PPAL CLl 


NUM'R TYPl 


F'IS'G DATA 


DISTRACTOR 


WhVPY 1 


DFPTd 2 


CONIHERSION 


VERBAL CLl 


DFPTU 7 


I'ERBAL CLl 


ITRBAL CLl 


ORDFR 2 


LENGTH 


VERBAL CL2 


DEPTH 2 


DEPTH 2 


NUM'R TYPl 


ORDER 2 


ORDER 2 


\mBAL GL2 


ORDER 2 


PRONOUNS 


PRONOUNS 


PRONOUNS 


PRONOUNS 


SEQUENT' L 


SEQUENT 


STEPS 


SF.OUT^NT'L 


HIERARCHY 


REUSE DATA 


ADJ'>'CfIVES 




STEPS 


HIERARCKY 


SEQUENT' L 


REUSE DATA 


ADJECTIVES 


ADJECTIVES 


REUSE DATA 


HIERARCHY 


REUSE DATA 


STEPS 


HIERARCHY 


ADJECTHTS 
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CHAPTER VI 



DISCUSSION Am SUGOESTIOWS 
FOR FURTHER RES EARCH 



The results of this study clearly indicate that the variable 
MEMORY 2 is highly significant for predicting the difficulty of 
word problems. By itself, the variable accounted for hl% of the 
variance. Its simple correlation with the dependent variable 
was a healthy .702. Furthermore, the variable was consistently 
first on all criteria of importance. The definition of MEMORY 2 
is composed of basically two subvariables : (a) the number of 
formulas to be recalled in pursuing the correct solution, plus 
the number of steps in each formula, and (b) the number of con- 
versions required to achieve the correct solution. For purposes 
of discussion, both parts can be combined in one word: recall. 

The findings in this study, then, indicate that for the 
Junior College students saaipled, an arithmetic word problem which 
requires some recall of previously learned facts is more difficult 
to solve than one that does not. Of course, there is the question 
"If the needed recall facts were made available to students, say, 
via a hint, could students then carry the problem to solution?" 
This is a meaningful research question. To give this question 
deserving attention in future research, the driving functions 
in- the software package used for this study have already been 
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"patched" to present hints and trap student response data pre and 
post hints. No comments will be made here regarding results since 
the data have not yet been organized into proper form. 

It is indeed encouraging and rewarding to discover that a 
memory-related variable is so potent. Suppes (1966, page 2hQ) also 
found that his memory variable was quite important in predicting 
the difficulty of fourth-grade addition problems. He goes on to 
state (p. 23k): 

From a psychological standpoint, the most 
suggestive single finding is probably the 
importance of the process variable NSTEPS, 
or of its component variables, particularly 
memory, in all the relevant analyses. . . 
If. . . the dominant variables had turned 
out to be magnitude variables, then a less 
significant first step would have been taken, 
because anyone would immediately ask what 
characteristics of the processing done 
internally by the students made these magni- 
tude variables so significant. In postulating 
process variables and being able to establish 
their direct importance, we have already 
been able to move part this first step. 

As stated at the beginning of this paper, the results under 
discussion are but one-sixth of a larger study, representing the 
author's doctoral dissertation at UCLA. Space does not permit 
a full discussion of all results in that dissertation, but it is 
most appropriate at this point to allow for a passing comment. 
Regression analyses were also run on subsets of the 172 problems 
(toward more heterogenous problem sets); on average latency of 
response, and average success latency for all 172 problems. 
Appendix I displays the analyses of the data unaccompanied by 
discussion. 
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Perusal of these data vill show that the variable MEMORY 2 is 
also of premiere Importance! 

Professor Suppes has eloquently pointed out the psychological 
importance of this variable. But there is another important point 
to be made. The variable MEMORY 2 is well defined I This fact 
makes MEMORY 2 a agmatic variable and ought to facilitate its 
implementation into the world of educational practice. 

This success is somewhat dulled, however, by the variable's 
multiplicity of definition. Is the recall of formulas what makes 
it a strong variable? Or is it the need to recall appropriate 
conversion units? A finer grained analysis was not performed 
in this study to help answer this question. Neither can an 
answer be derived from the present results since "pure" conversion 
and formula variables were present in the set X of structural 
variables (CONVERSION, and MEMORY l). Hopefully, future research 
will answer this question. 

The variable ORDER 1 showed a respectable amount of robustness. 
It was second on three of the four criteria of importance, includ- 
ing the stated definition of importance used in this study. The 
variable's showing should not be disappointing even to those who 
abide by the beta weight as an index of importance. Here, ORDER 1 
was fifth (see Table 1, Chapter V). The siinple correlation of 
ORDER 1 with the dependent variable, the percent of correct responses, 
was .1*78. 

The implication of the ORDER 1 variable to problem solving 
difficulty is that students did not "internalize" problems. 
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Item analysis relating to this variable showed that often students 
knew a problem should be solved by division. But when the correct 
answer was to be achieved by conmanding the computer to perform 
A f B, very often they asked the machine to perform B -f A, and 
promptly lablelled the latter as the correct answer. Again there 
appears a dichotcsmous query: "was this carelessness, or a more 
deep pedagogical reason?" 

In any event, note this is another well defined variable 
and thus easily implementable into educational practice. This, 
together with the fact that the set ^MEMORY 2, ORDER 1^ alone 
accounts for 57^ of the variance, makes for a significant contribution 
to predicting problem solving difficulty for Junior College arith- 
metic students. 

It is time to turn to the unpleasant task of discussing suppresor 
variables. The next two variables, NOUNS and DEPTH' 1 set the stage. 
Both have positive simple correlations with the dependent variable 
(.301 and .k86, respectively); both are, by definition and other 
criteria, important; yet both have significantly high negative 
regression coefficients— and, therefore, negative beta weights. 

It is tempting to offer the explanation: more nouns and 
deeper sentences reduce the difficulty of arithmetic word problems. 
But this would be a spurious interpretation in view of the 
positive, simple correlations. It is more sensible to say that 
because of the inosrcorrelatedness of the independent variables, 
the regression procedure, in trying to minimize the sum of squares, 
achieves this goal by assigning negative coefficients to these two 
variables. 
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This conclusion is further supported by glancing at the simple 
correlations between variables. The correlation between NOUNS and 
LENGTH was .9611 Furthermore, NOUNS was entered into the regression 
equation immediately after LENGTH. 

DEPTH 1 was correlated with VERBAL CLUE 2 (.576), MEMORY 2 (.561), 
and NOUNS (.731). All these variables entered the regression before 
DEPTH 1. 

Before we lower the life boats, it should be noted that the ship 
may be listing, but still floating. The goal in this study was to predict 
problem solving difficulty. This can be accomplished even with the 
appearance of negative regression coefficients. If the concern was with 
the more delicate question of cause-effect, then the appearance of 
negative weights would be more damaging. 

The variable LENGTH, though not as robust as the other four, was high 
on the list of predictive worth. Its simple correlation with the de- 
pendent variable was .382. This correlation is too small to venture a 
cause-effect relationship. But with the other variables present, it can 
be said that the length of a problem does affect its difficulty. 

Recall that for the students involved in this study, mathematics 
has been a difficult subject. These are people who after twelve years 
of formal schooling, are not able to perform at a level of eight, or at 
most ninth grade arithmetic. Word problems in particular, by record and 
the student's own admission, are particularly difficult. In view of this, 
it's not hard to understand why a lengthy problem, with unordered data, 
and requiring recall of previously learned facts, has a tendency to — 
if we may use the current vernacular — pshyc the students out. 
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Again, note that the length variable is the most well 
defined of the entire set X, of structural variables, and the 
easiest to measure. Such an asset should prove valuable in its 
implementation into practice. 

The first five variables of the ordered set X' have been 
discussed. Their robustness and highly acceptable definitions 
were encouraging in the effort to predict problem solving difficulty 
for Junior College arithmetic students. It voui.d not be meaningful 
to belabor the discussion to the remaining 25 variables, for their 
intercorrelatedness would make the interpretation rather artificial. 
The interested reader is referred to Table k. Chapter V, the other 
tabulated data, and Appandix 1. These are presented clearly enough 
for anyone to reach his own conclusions. In the author's opinion, 
the accomplishment of the study rests with pinning down the set 
[memory 2, ORB.ER ij whose two elements are structural variables 
in predicting problem solving difficulty of CAI-presented word 
problems for Junior College arithmetic students. 

Before concluding this section, it will be informative 
to summarize in tabular form these results along with those in 
previous structural variables studies. The latter have been already 
analyzed in the review of the literature chapter, thus there is 
no need for further comment here. Table 5 presents this summary. 
Let the reader beware that the table is highly artificial. Place, 
time, subjects, variables, and criteria of importance differ in 
each study. 
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Thus comparisons should be made. . .carefully . if at all. The inability 
to make valid comparisons from data like in Table 5 brings to attention 
a sad state of affairs in this young branch of educational research. A 
certain amount of standardization is sorely needed. Criteria of import- 
ance, for example, ought to be the same or equivalent for all structural 
variables studies. A first step toward this goal was taken in the present 
study by listing variables according to four criteria of importance. 
A pool of varibles and problems ought to be made available to researchers 
interested in this field. For CAI oriented studies, similar software 
ought to be uned. Toward this end. the APL software used in this study 
is available to anyone who wishes it. Appendix 2 presents only some of 
the functions, for illustrative purposes. 

Furthermore, research studies are needed to identify, define, 
and categorize structural variables. From these studies, of factor 
analytic nature, more orthogonal variables can be discovered. Also, 
more measures of dependent variables are needed; for problem solving 
difficulty is not only reflected by percent of correct responses. Suppes. 
Hjonan. and Jerman (1966). for example, analyzed average success latency. 
In the present study, of which this paper is only a part, average total 
latency and average success latency were used as dependent variables. 
In this same study, percent of correct responses together with average 
total latency were used in a canonical correlation analysis. In the 
author's opinion, canonical analysis offers real possibilities for these 
types of studies. 

Perhaps the vehicle to be used in achieving, normalization of 
process is a structural variables Journal-aware though the author is 
of the present proliferation of journals. 
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APPENDIX 1 



Tables and graphs of other regression analyses related to 
the results in this paper. These are offered without further 
comments, save that three regression runs are represented. 



TABLE 9 



REGRESSION COEFFICIENTS, STANDARD ERRORS, COmJTED T-VAimS 
AND R-SQUARED DROP CAUSED BY THE REMOVAL OF EACH VARIABLE IN 
TURN FOR OBJECTIVE 3 ( 72 PROBLEMS, AND 30 VARIABLES ) 



VARIABLE 


REGRESSION 
COEFFICIENT 


STANDARD 
ERROR 


COMPUTED 
T-VALUE 


RSQ 
DROP 


0 CONSTANT 

1 OPERATIONS 

2 DIVISION 

3 HIERARCHY 

4 STEPS 

5 LENGTH 


"2.48986 
-0.0268 
-0.0302 
0.0053 
0.0177 
0.0080 


0.0667 
0.1936 
0.0228 
0.1001 
0.0040 


-0.4011 
-0.155S 
0.2318 
0.1764 
1.9851 


0.0010 
0.0002 
0.0004 
0.0002 
0.0172 


6 DrPK/ 1 

7 ZJffPT^ 2 

8 VERBS 

9 ADJECTIVES 

10 /V0i/i\?5' 


-0.0095 
0.1467 
0.0825 
0.0000 

-0.0949 


0.0070 
0.2509 
0.0590 
0.0000 
0.0359 


-1.3592 
0.5848 
3.3973 
1.0000 

-2.6402 


0.0095 
0.0018 
0,0104 
0.0000 
0.0369 


11 ADVERBS 

12 PRONOUNS 

13 NOUNS •A'ERBS 

14 NOUNS '.ADJECTIVES 

15 PRONOUNS '.NOUNS 


0.0294 
-0.0674 
0.1263 
0.0293 
2,1116 


0.1211 
0.0828 
0.1306 
0.0324 


0.2424 
-0.8135 
0.9674 
0.9025 

1 • OU DO 


0.0004 
0.0035 
0.0050 
0.0022 
0.0036 


16 VERBS '.ADVERBS 

17 SEQUENTIAL 

18 1 

19 C'i?Z)£'i? 2 

20 y&PSA& (7LW; 1 


0.0230 
0.0278 
0.0378 
-0.0153 
0.2190 


0.0520 
0.2199 
0.2459 
0.1230 
0.2623 


0,4416 
0.1265 
0.1535 
-0.1247 
0.8351 


0.0011 
0.0001 
0.0002 
0.0001 
0.0038 


21 7S?S/{i CLW; 2 

22 DISTRAC'CORS 

23 SUPERFLUOUS DATA 

24 MISSING DAZ'A 

25 i?^'^!) 2.142V1 


0.0532 
0.1426 
0.0204 
0.4885 
0.1679 


0.1397 
0.1298 
0.1944 
0.3254 
0.2620 


0.4523 
1.0986 
0.1049 
1,4968 
0.6410 


O.OOIC 
0.0063 
0.0001 
O.OilO 
0.0023 


26 mssi^ rJPii 1 

27 ii^i/ftSFi? fJPi? 2 

28 CONVERSION 

29 Affi^/Oi?! 1 

30 i^s'.'-my 2 


0.118G 
0,3488 
-0.3409 
0.1316 
0.2133 


0.21iC 
C.1886 
0.2712 
0.1003 
0.0721 


0.5521 
1.8489 
-1.2569 
1.3123 
2.9562 


0.0017 
0.0177 
0.003U 
0.0090 
0.0462 
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TABLE 10 



SEQUENTIAL APPEARANCE OF THE VARIABLES INTO THE 
REGRESSION EQUATION FOR OBJECTIVE 3. ALSO SHOWN 
ARE THE MULTIPLE R, R-SQUARED, AND THE INCREASE 
IN R-SQUARED (72 PROBLEMS, AND 30 VARIABLES) 



VARIABLE 



MULTIPLE 
R 



R 

SQUARED 



INCREASE 
IN RSQ- 



30 MEMORY 2 
18 ORDER 1 

21 VERBAL CLUE 2 
15 PRONOUN'.NOUN 

12 PRONOUNS 

22 DISTRACTOR 

27 NUMBER TYPE 2 

13 NOUN '.VERB 

14 NOUN '.ADJECTIVE 

24 MISSING DATA 

28 CONVERSION 
11 ADVERBS 

6 DEPTH 1 
5 LENGTH 
10 NOUNS 

8 VERBS 

29 MEMORY 1 

20 VERBAL CLUE 1 

25 REUSED DATA 

26 NUMBER TYPE 1 

7 DEPTH 2 

1 OPERATIONS 

16 VERB '.ADJECTIVE 

3 HIERARCHY 

2 DIVISION 

17 SEQUENTIAL 

4 ^rFP^ 
19 Oi?Z?£7? 2 

23 SUPERFLUOUS DATA 
11 ADJECTIVES 



0.6453 
0.7307 
0.7583 
0.7755 
0.7898 



0.4164 
0.5339 
0.5750 
0.6014 
0.6238 



0.4164 
0.1175 
0.0411 
0.0264 
0.0224 



U •7997 


0.6395 


0.0157 




0.6532 


0.0137 


0,8193 


0.6713 


0.0181 


0,8252 


0.6810 


0.0097 


0,8280 


0.6856 


0.0046 


0.8329 


0.6937 


0.0081 


0.8361 


0.6991 


0.0053 


0.8403 


0.7061 


0.0070 


0.8521 


0.7261 


0.0200 


0.8642 


0.7468 


0.0208 


0.8680 


0.7534 


0.0066 


0.8741 


0.7641 


0.0106 


0.8765 


0.7683 


0.0042 


0.8785 


0.7718 • 


0.0035 


0.8793 


0.7732 


0.0014 


0.8803 


0.7749 


0.0018 


0.8810 


0.7762 


0.0012 


0.8817 


0.7774 


0.0012 


0.8818 


0.7776 


0.0002 


0.8819 


0.7777 


0.0002 


0.8819 


0.7777 


0.0002 


0.8820 ' 


0.7779 


0.0002 


0.8820 


0.7779 


0.0001 


0.8820 


0^7779 


0,0001 


0.8820 


0.77dO 


0.0000 
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TABLE 12 



ORDER OF It-iPORTANCE OF THE 30 STRUCTURAL VARTABLES AS 
MEASURED BY FOUR CRITERIA OF IMPORTANCE. THE FIRST OF 
THESE IS I'HE CRITERION OF IMPORTANCE AS MEASURED BY 
THE DEFINITION^ THE SECOND IS THE ORDER OF APPEAMNCE 
OF THE VARIABLES INTO THE REGRESSION EQUATION^ THE 
THIRD y BY THE DECODED BETA WEIGHTS, AND THE FOURTH 
BY THE SIZE OF THE PARTIAL CORRELATION COEFFICIENT 
{VARIfBLES HAVE BEEN ABBREVIATED) 



DEFINITION OF ORDER OF DECODED BETA PARTIAL COR"!. 

OF IMPORTANCE APPEARANCE WEIGHTS COEFFICIENT 



MEMORY 2 


mWRY 2 


NOUNS 


MEMORY 2 


mUNS 


ORDER 1 






NUM'R TYP2 


VERBAL CL2 


DEPTH 1 


X'ERBAL CL2 


LENGTH 


WON I NOUN 


VERBS 


OPERATIONS 


MIS^G DATA 


PRONOUNS 


MEMORY 2 


PRON-.NOUN 


Tf 111 >T^ ^ 

VERBS 


DISTRACTOR 


MEMORY 1 


VERB lADVRB 


DtJrIH 1 


NUM^R TTP2 


MIS • G DATA 


NOUNS 


MEMORY 1 


■ NOUN '.VERB 


PRON-.NOUN 


DISTRACTOR 


CONVERSION 


PRONiADJTS 


NUM^R TYPl 


DEPTH 1 


DISTPACTOR 


MIS^G DATA 


CONVERSION 


Nin-im TYP2 


NOUN -.VERB 


CONVERSION 


NOUN '.VERB 


SUP'S DATA 


VERBAL CLl 


ADVERBS 


PRONOUNS 


ADVERBS 


PRON-.NOUN 


DEPTH 1 


DISTRACTOR 


PRONiADJTS 


PRONOUNS 


LENGTH 


PRON'.ADJTS 


NOUN I VERB 


REUSE DATA 


NOUNS 


VERBAL CLl 


WRBS 


PRONiADJTS 


VERBS 


REUSE DATA 


MIS'G DATA 


DEPTH 2 


mWRY 1 


VERBAL CL2 


DIVISION 


NUM^R TYPl 


VERBAIj CLl 


OPERATIONS 


CONVERSION 


VERB lADVRB 


REUSE DATA 


DEPTH 2 


W,RBAL CLl 


OPERATIONS 


NUMH^ TYPl 


NUM'R TYPl 


MEMUR" 1 


VERBAL CL2 


DEPTH 2 


VERB-.ADWB 


DEPTH 2 


HIERARCHY 


OPEPATIONS 


ADVERBS 


LENGTH 


ADVERBS 


VERB-.ADVRB 


STEPS 


NUM'R TYPl 


DIVISION 


HIERARCHY 


HIERARCHY 


ORDER 2 


STEPS 


DIVISION 


ORDER 2 


PRONOUNS 


ORDER 1 


SEQUENT^ L 


DF/ISION 


SEQUENT 'L 


SEQUENT^ L 


STEPS 


ORDER 1 


STEPS 


ORDER 2 


ORDER 2 


SEQUENT^ L 


REUSE DATA 


SUP^S uATA 


SUP^S DATA 


SWF'S DATA 


HIERARCHY 


ADaiCTIVES 


ADJECTIVES 


ADJECTIVES 


ADJECTIVES 



TABLE 13 



EECRESSION COEFFICIENTS, STANDARD EERORS, COMPUTED T-VALVES 
AND E-SQUARED DROP CAUSED BY THE REMOVAL OF EACH VARIABLE IN 
TURN FOE OBJECTn^ if ( 172 PEOBLEf-fS , AND 30 VARIABLE) 



VARIABLE 



REGRESSION 
COEFFICIENT 



STANDARD 
ERROR 



COMPUTED 
T-VALUE 



ESQ 
DROP 



CONSTANT 

OPERATIONS 

DIVISION 

HIERARCHY 

STEPS 



0 
1 

3 

5 Ln\GTH 

C DEH'H 1 

7 DEPTH 2 

8 VERBS 

9 ADJECTIVES 

10 NOUNS 

11 ADVERBS 

12 PRONOUNS 

13 NOUNS: VERBS 

14 ; ADJECTIVES 

15 PRONOUNS -.NOUNS 

16 VERBS lADVERBS 

17 SEQUENTIAL 

18 mffP 1 

19 (J/fPI^? 2 

20 y£'i-?S4I cz,r-v 1 

22 DI3TPA.CI0RS 

23 SUPERFLUOUS DATA 
2if MISSING DATA 

25 .WyOT D/ir/l 

26 A'^/^^?:? J-ITF 1. 

27 miFFR riPi- 2 
23 CONVERSION 

29 iVr/VCifl 1 

30 MEMORY 2 



1.815'-;5 



1.3915 


2.0236 


0.6876 


0.0008 


11.2315 


6.5496 


1.7148 


0.0050 


0.2804 


0.7035 


0.3985 


0.0002 


-2.^-272 


4.0291 


-0.602i' 


0.0006 


-0.1236 


0.13UO 


-0 .9160 


U . U Jl H 


0.1255 


0.1400 


0.8967 


0.0003 


-1,9U87 


6.8071 


-0.2803 


0.0001 


1.2650 


2.1774 


0.581C 


0.0005 


-0.5091 


1.4608 


-0.4033 


0.0002 


2.056? 


1.3C3i 


1.5085 


0.0036 


6.2'!.36 


3.7602 


1.6604 


0.0047 




1.7534 


-1.6200 


0.0010 


1.7990 


3.5783 


C.5030 


0.00C4 


-2.8388 


2.0026 


-1.4176 


C.0034 


O.OOOG 


O.OOOC 


1.0000 


0.0000 


-0.i^220 


1.7973 


-0.2348 


0.0000 


-8.1001 


5.7798 


-1.4014 


0.0C33 


11.5196 


7.309 2 


1.5760 


0.0042 


-2.70S0 


4.2556 


-0.5365 


O.OOOf 


17.557i| 


7.60?j 


2.30^'^? 


0.0090 


-0 .9*129 


3.2865 


-0.2869 


0.0001 


10.8660 


4.3330 


2.5077 


0.0106 


7.4960 


7.8954 


0.9U93 


0.0014 


5.16i^3 


10.5370 


0.4901 


0.0003 


24.1036 


9.0717 


2.6570 


0.0120 


-6.4511 


7.2375 


-0.8913 


0.0013 


17.2618 


8.5853 


2.010'c. 


0.0069 


-1.5282 


10.2591 


-0.1400 


O.GOCO 


9.0147 


3.2847 


2.7444 


0.0128 


5.8064 


2.5960 


2.6219 


0.0117 



ERIC I 



SEQUdN"lAT- APPEAUANCS OF THE VABTABLE^i lU-JO 'JsF 
Fj'GmBSIOl^ ECUATlOIv FOB OB.lECTIVB U. ALSO SHOW 
ARE THE MULC'TPLE R-SQUAMD, AND T'-IE HCREASE 
IN B-SQUARFD (172 PROBLEhiS, AND 30 VAP.UBLES) 



VARIAhLti* 


MULTIPLE 


R 


INCREAS. 




R 


SQUARED 


IN ESQ 


29 [4tMuHX 1 


0.6853 


0.4696 


0.4695 


30 MEMORl 2 


0.7570 


0.5730 


0.'^.034 


13 OBVER 1 


0.7875 


0.6202 


0.0471 


2 DIVISION 


0.8077 


0.6524 


0.0322 


11 


0.8248 


0*6803 




25 EEUSED DATA 


0.8349 


0.f)971 


0.0168 


22 DISThACTORS 


0.S438 


0.7120 


0.0149 


10 i/JUNS 


0,8409 


0.7206 


0.0086 


27 NUMBEi-: TxPE 2 


0.8527 


0.7271 


0.0065 


23 SUPERFLUOUS DATA 


0.8558 


0,732'l 


0 .0053 


12 FEONOLfJS 


0.858i^- 


0.7369 


0.0C45 


20 VERBAL CLUB 1 


0.861^:. 


0. 71120 


0.0C52 


17 bEQlEhTIAL' 


0.8634 . 


0.7h55 


0.0034 


14 nCUNS lADc'ECTWES 


0.8647 


0.7U77 


0.0022 




0.8659 


0.7498 


0.0021 


5 Z?Z7Pr/f 1 


0.8668 


0,7513 


0.0016 




0.8676 


0.7527 


0*0014 


1, OPERATIOUS 


0.8682 


0.7538 


0.0010 


26 //wissi? tip:? i 


0.8689 


0.7550 


0.0012 


19 ORDER 2 


0.86S2 


0.7555 


0.0005 


24 MISSING DATA 


0.8693 


0.7557 


0.0002 


3 HITRARCHY 


0.8694 


0.7559 


0.0002 


9 ADJECTIVES 


0.8696 


0.75'52 


0.0003 


7 DSPT/i 2 


0.8696 


0.7bi.. 


0.0001 


8 


0.8697 


0.756'l 


0.0002 


13 NOUNS iVERBS 


0.8699 


0.7567 


0.0003 


21 K£7?ML 2 


0.8700 


0.7509 


0.0002 


IS VERBS wlDJECTIVES 


0.8700 


0.7569 


0.0001 


28 CCNl^RSION 


0.8700 


0.7559 


0.0000 


jp. PEONOUSt NOUNS 


0.8700 


0.7559 


0.0000 



inpuiainntnatnnin 



TABLE 15 



ORdEP: OF I!m)RTANCE OF TEE 30 STRUCTURAL VARIABLES AS 
h{EASUR3r> BY FOUR CRITERIA OF IMPORTANCE. T'iE FIRST OF 
THESE IS TrlE C^TTFRION OF IMPORTANCE A9 mASURED BY 
THE dKFIUITIOh\ THF. SECOW IS THE ORDER OF APPEARANCE 
or THE VARIABLES INTO THE REGRKSSION WUATION, THE 
THIRD, BY THT- DECODED BETA WEIGHTS, /J^'D THE POUHTIf 
BY THE SIZE OF THE PARTIAL CORRELATION COEFFICIENT 
(.VARIA3LES HAVE BEEN ABBREVIATED) 



DEFINITION OF 
OF IMPORTANCE 



ORDER OF 
/iPPEARANCE 



DECODED BETA 
WEIGHTS 



PARTIAL CORR, 
COEFFICIENT 



MEMORY 1 
REUSE DATA 
MkWRY 2 
DISTRACTOR 
VEird/lL CLl 



MEMORY 1 
MEMORY 2 
ORDER 1 
DD/ISION 
ADVERBS 



REUSE DATA 
NUMKR TYP2 
VERBAL CLl 
ORDER 1 
DF/ISION 



MEWRY 1 
MEMORY 2 
ORDER 1 
DIVISION 
ADVERBS 



NUM^R TY?2 
DIVISION 
/iDVERBS 
ORDER 1 
NOUNS 



IIBUSE DATA 
DISTRACTOR 
NOUNS 

NUM'R TYP2- 
SUP^S DATA 



SUP^S DATA 
MIS'C DATA 
DISTRACTOR 
SFQUFNT'L 
NUM^R TYPl 



REUSE DATA 
DISTRACTOR 
NOUNS 

NUMKR TYP2 
W'JRBAL CLl 



PRON:ADJTS 
SEQUENT^ L 
LENGTH 
SUP'S DATA 
NUM'R TYPl 



PRONOUNS 
]'ERBAL CLl 
SEQUENT 'L 
PRONxADJTS 
LENGTH 



MEMORY 1 
ADVERBS 
MEMORY 2 
CONVERSION 
DEPTH 2 



SUP'S DATA 
PRONOUNS 
SEQUENT' L 
PRON'.ADJTS 
LENGTH 



PRONOUNS 
OPERATIONS 
STEPS 
ORDER 2 
VERBS 



DEPTH 1 
STEPS 
OPERATIONS 
NUM'R TYPl 
ORDER 2 



ORDER 2 
STEPS 
NOUN '.VERB 
IRONxADJTS 
PRONOUNS 



DEPTH 1 
STEPS 
OPERATIONS 
NUM'R TYPl 
ORDER 2 



NOUN '.VERB 
DEPTH 1 
MIS'G DATA 
HIERARCHY 
ADJECTB'ES 



MIS'G DATA 
HIERARCHY 
ADJECTIVES 
DEPTH 2 
VERBS 



VERBAL CL2 
OPERATIONS 
NOUNS 
WRBS 

adject.:vj,s 



NOUN: VERB 
MIS'G data 
ADJECTIVES 
HIERARCHY 
VEP3AL CL2 



DEPTH 2 
VERBAL CL2 
PRON'.NOUN 
VFRB'.ADVRB 
CONVERSION 



NOUN '.VERB 
VERBAL CL2 
VERB lADWB 
CONnRSION 
PRON'.mUN 



VERB'.AVVRB 
HIERARCHY 
DEm: 1 
LENGTH 
PRON'.NOUN 



DEPTH 2 
VERBS 

VERB'.ADVRB 
CONVERSIOIJ 
PRON'.NOUN 



TABLE 17 



HEJHSSSICN COEFFICIENTS, STANPAHD EnHOR<^\ rO'WTfn t-^'a 
AND R-SQUARED DROP CAUSED BY TI!P. PEMOVAJ: OF PACF VARrM 
TURN FOR OBJECTIVE 5 ( 172 PROBLFf-iS, AND 30 VAPIAPLE 

RFGRE'SSION STANDAPn rmmirpvn 





REGRESS..ON 
COEFFICIENT 


STANDARD 
ERROR 


COhfPUTED 
T-VALUS 


RSQ 
DWP 


0 CONSTANT 

2 DIVISION 

3 HIERARCHY 

4 STE'PS 

5 LENGTH 


"7.87572 
-4.2436 
29.422'' 
-0.7129 
1.7242 
0.0000 


4.3395 
14.4630 
1.5316 
8.5338 
0.0000 


-0.977P 
2.0342 

-0.4P55 
0.2020 
1.0000 


0.0018 
0.0080 
0.0004 
0.0001 
G.OOOO 


7 DEPTH 2 

8 '/BEDS 

9 ADJECTIVES 

10 A^0i/.V5 


-0.1096 
-15.4948 
9.1322 
0.0000 

-4.5518 


0.2949 
15.0388 
4.6566 
0.0000 
2.?356 


-0.3717 
-1.0303 
3.9611 
1.0000 
-1.9532 


0.0003 
0.0021 
0.0070 

o.ooon 

0.0045 


12 PRONOUNS 

13 NOVNS '.VERBS 

14 NOUNS :AUJ3CTIITS 

15 PRONOUNS I NOUNS 


15,5051 
8.5458 
10. r- 61 5 
-7.2293 
-127.5630 


7.9277 
7,3464 
7.8996 
3.0029 
no. 6954 


1.9'-^58 
1.1633 
1.4237 
-2.4075 
-1.4065 


o.oor>8 

0.0027 
0.0037 
0.0050 
0.0039 


16 inSPBS xADWlBS 

17 SEQUENTIAL 
19 Oi?f/ffi 1 

19 OiryS'/? 2 

■20 '/E/.'B.-iL 6'Lf;ff 1 


-0,7102 
-9.6372 
26.4054 
1.2532 
29.9320 


3.9182 
12.5878 
16.0919 

'^.2653 
16.9353 


-0.1812 
-0.7F56 
l.RiJOq 
0.1353 
1.7673 


0.0001 
0.0011 
0.')053 
0.719n 
0.0061 


21 P^/ffl/li CLi/P 2 

22 DISTRACTORS 

23 SUPERFLUOUS DATA 

24 MISSING DATA 

25 /?£'i/5iT 


4.6386 
8.9944 
-8.7771 
28.3436 
31.9389 


7.2309 
9.4294 
17.3760 
23.1524 
19.8707 


0.64?. 5 
0.'J539 
-0.5051 
1.2242 
1.6C73 


G.00C8 
0.0017 
0.0005 
0.0029 
0.0050 


26 NU.'SER TYPE 1 

27 MMS-i? Stirs' 2 

28 CONmiSION 

29 /^ffiA/QPI 1 

30 /«/?J 2 


-11.7147 
62.8473 
15.5708 
12.5833 
28.6438 


15,5997 
18.9060 
22,6403 
7.1886 
5.7153 


-0.7510 
3.3242 
0.6877 
1.7505 
5.0117 


0.0010 
0,0215 
0.0010 
0.0059 
0.0489 



TABLE 13 



SEQVUmAL APPEATiASCB OF THE VAIilAELES WTu THE 
REGRESS TOP EQUATIOH FOR OBJECT.V/E 5. ^.ifC SHOWtl 
AF:E THE hiULTlFLE R, R-SQUARED, AIW THE IVCPEflSF 
m R-SQUARED (172 RROBLEIIS, AKO 30 VAP.IABIES) 



VARIABLE 


WLTIPLE 


R 


INCRl 




R 


SQUARED 


IN R. 


30 lEMORx 2 


O.KSr:2 


0.4666 


0.456T 


1 uFEEATICNc* 


0.7336 


0.5382 


0.0714 


27 NUrDlH TjFF 2 


0.7G'44 


0.5843 


0.0463 


11 ADVERhS 


0.7902 


0.6244 


0,0401 


21 CZiW' 2 


0.8067 


0.6r>08 


ft ft',»R'^ 


18 ORDER 1 


o.eion 


0.6665; 


0.0157 


2*^ MISSWG . 


0.6226 


0.6767 


0.C102 


I'l iVf^i 77o : ADJECTl'VFS' 


0.8263 


C.6328 


O.OOCl 


2 Dj. VIS 101! 


0.8295 


0.6881 


0.0053 


29 iWOI 1 


0.8334 


\J • LI J *T V 


u • UUOt> 


20 VERBAL CLUE 1 


0.8354 


0.6979 


O.OOi53 


7 DErTh 2 


0.8372 


O.'^OOO 


0.0030 


2r DISTRACTORS 


0.8381 


0.''024 


0.0015 


25 REUtED DATA. 


0.8-92 


0.7043 


0.0018 


J.U liUUlvO 


0,8401 


C.7058 


0.0015 




0.8414 


0.7080 


0.0C22 


13 mUNS '.VERBS 


O.P'.TjS 


0.7112 


0.0032 


17 SEQUENTjM, 


0.3'^4i, 


0.7130 


0.0019 


26 MAffiffi? 2TPE 1 


0.8451 


0.7142 


0.0012 


15 PROh'OUNS-JJOUNS 


0.84b7 


0.7152 


o.ocio 


12 PRONOUNS 


0.8468 


0.7171 


0.0019 


28 CONVl-RSIOt-: 


0.K474 


U.7181 


0*0010 


2? SUPERFLUOUS DATA 


0.8479 


0.71V9 


0.0C0& 


3 HlfPJRaiY 


0.8482 


0.7 5 94 


o.oooi; 


6 i^-s^a'^ 1 


0.8483 


0.7lSb 


0.0002 




0.8484 


0.7198 


0.0002 


15 KH?B5:/li7JrC2'IF£5 


0.8484 


0.7198 


0.0001 


28 C'i?Z?S? 2 


0.8485 


0,7200 


0.0001 


5 iF//ffra 


0.8485 


0,7200 


o.ooco 


9 ADrmCTlVES 


0.8485 


0.7200 


0.0000 



TABLE 20 



ORDER OF Ii.rPORTAi:CE OF THE 30 STRUC'VPAL VARIABLES A£ 
MEASURED BY FOUR CRITERIA OF P.fF-CRTANCE. Tl<E FIRS"^ OF 
THESE IS THE CRITERION OF It-IFORTANCE As' t-fFASURED BY 
THE DEFINITION, THE SECOND IS THE ORDER OF APPEARANCy 
OF THE VAJHABLES INTO THE REGRESSION EQUATION, ' TEE 
THIRD, BY THE DECODED BETA WEIGHTS, AND THE FOURTH 
BY THE SIZE OF THE PAJiTLlL CORRELATION COEFFICIENT 
(VAJflABLYS HA^/E BEEN ABBREIWED) 



DEFINITION OF 
OF IMPORTANCE 



ORDER OF 
APPE/JIANCE 



DECODED BETA 
WEIGHTS 



Partial corr. 
coefficient 



ORDER 2 
l-IEMORY 2 
NUM'R TYP2 
niviSION 
VERBS 


MEMORY 2 
OPERATIONS 
NUM'R TYP2 
ADVERBS 
VERBAJi CL2 


MEMORY 2 
VERBS 
NOUNS 
MEMORY 1 
'm'R TYP2 


l-fEMORY 2 
OPERATIONS 
NUM'R TYP2 
nUvbr.ab 
i^RBAL CL2 


ADVERBS 
VERBAL CLl 
I'EMORY 1 
ORDER 1 
PRON'.AdJTS 


ORDER 1 
MIS'G DATA 
PRON'.ADJTS 

MEMORY 1 


ADVERBS 
PRON'.ADJTS 
DIVISION 
PRON'.NOUN 
NOUN: VERB 


ORDER 1 
MIS'G DATA 
miORY 1 
PRON'.ADJTS 
DIVISION 


REUCE DATA 
NOUNS 
PRONiNOUN 
NOUN '.VERB 
MIS'G DATA 


VERBAL CLl 
DEPTH 2 
DISTRACTOR 
REUSE DATA 
NOUNS 


PRONOUNS 
ORDER 1 
REUSE DATA 
VERBAL CLl 
MIS-G DATA 


VERBAL CLl 
NOUN '.VERB 
DEPTH 2 
ITT^BS 
SEQUENT'L 


PRONOUNS 
DEPTH 7. 
OPERATIONS 
DISTRACTOR 
SEQUENT' L 


VERBS 
NOUN'.ITRB 
SEQUE'.T'L 
NU.'i'p TYl'l 
PRON-.fJOUN 


OPERATIONS 
DEPTH 2 
DISTRACTOR 
CONVERSION 
VERBAL CL? 


RBJSE DATA 
PRONOUNS 
DISTRACTOR 
NCVNS 
NUM'R TYPl 


NUiVn TYFl 
CONVERSION 
VERBAL Cl2 
SUP'S DATA 
HIERARCHY 


PRONOUNS 
CONVEtlSION 
SUP'S DATA 
HIERARCHY 
DEPTH 1 


NUM'R TYPl 
DEPTH 1 
SEQUENT' L 
SUP'S DAT.l 
HIERARCHY 


PRON'.NCUN 
CONVERSION 
SUP'S DATA 
HIERARCHY 
DEPTH 1 


DEPTH 1 

STEPS 

VERB;ADVHB 

LENGTH 

ADJECTIVES 


STEPS 
I'ERS'.ADl'RB 
ORDER 2 
LENGTH 
ADJECTIinrS 


STEPS 
VERD',ADIT.B 
ORDER 2 
LENGTH 
ADJECTIVES 


STEPS 
VERF'.ADVRB 
ORDER 2 
LJINGTH 
ADJECTIVES 



APPENDIX 2 



COMPUTER ASSISTED PROBLEM SOLVING ( C A P S ) AT 
GOLDEN TOST COLLEGE 



AKGELO SEGALLA 



The CAPS pi-ogrnm wuh designed to augment the nac^-enatics rurricuJum 
at Golden Wast ColJcKe. Its existence is a result of a mutual interest 
on !:!ie parts of the collcRe administration and the ir.atho.n,atics staff to 
fost..r innovation in the learning process. CAPS v.-as supported by foi:r W 
general objectives: 

1. To appropriate the coaputer in the curriculum as an innovative 



tool^ 



— . To nrovi df ■in*:M-«irrrt>-«.- t.r-> ^^ „ i.... .i. ,..»-■> <- 

^ v.x,.,.^.ic<.u t-uic-njLCJLe ut eacn scuaent s 

path through the progra:P. reflecting his thought patterns on each vord problcn; 

3. To provide instructors with an item-analysis option of each word 
problem in the prograni; and 

^. To identify and define structural variables for each problem in an 
effort to predict the difficulty level of the problem (difficulty icvcl 
derived from regression analysis) . 

The CAPS program is in many ways similar to the pioneering CAI work 
conducted by Professor Suppes at Stanford University (Suppes, Jerman and 
Bryan, 1968), but it does offec several important differences, reflecting the 
fact that it wa.s coded independently of the Stanford materials. The most obvious 



diiCercncc is else laiiguciRc: CAPS was coded in APL, the Stanford program is 
in UASIC. Further, the analysis funoclons, the heart of the CAPS program, 
seem to be a great deal more sophisticated in their ability to Crap even the 
most minute bits of response data, such as the number of seconds it takes 
student T. to nccomp] Lsh step J in problem K, and the exact steps he used 
in achieving Uia answer. The purpose here is nor. to compare the two programs, 
especially in view of the fact that our knowledge of the Stanford materials is 
limited to the information published by Professor Suppes and his associates, and 
these publications usually do not refer to the internal workings of the programs. 

HARDWARE 

The problems in CAPS are presented to and solved by students via IBM 2741 
terminals located in the Multi-Media Center and the Math-Science building at 
Golden West College. Thirty-four (34) terminals are available to students 
daily from eight a.m. to ten p.m. The terminals are hardwired to an IBM 370/155 
computer located ten miles away at Orange Coast College, Costa Mesa, California. 
Orange Coast College is the "sister" college to Golden West, and both comprise 
the Coast Community College District. 

SOFTWARE 

The CAPS program is coded in APL, a new, interactive language, highly 
mathematically oriented, yet readily adaptable to computer assisted instruction; 
a spinoff not anticipated by the creator of the language. Dr. Kenneth Iverson, 
presently with IBM. Mr. John R. Clark, CAI consultant to the Coast Community 
College District, codfd the functions with direction from the author. F.<^u.r«i 
4iic.inyr. ^ of the thirty-six (36) functions used in the CAPS program. 



vAt.Ci!.:crX 
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[13] ->-oxi~.;-r-v/;.?7 

[lU] 'YOU AlVtl no? ALLOICD SPACES ARO'JiW THE ' '.mEH P.EPRESEi'TIUG FRACTIONS. 

[15] • •^•[l+.V'O 

[16] 1,'iS 

[17] fli-/./!.'; FimCTIOII CnEC}(S STUDEUT IllPUT FOR ALGEDRAIC CORRECTHESS 

[18] nOPFRATOnS BET.IEKU OFERAtWS 

[19] (\mTCUED PARENTHESIS ETC 

y 

VAPOTJSnLY]^ 
7 W*-APOLISil I -J, 

[1] fM(~iV' ( )• )/J)[4.1,,<9(2,pfOp/. J-I/ef )A/^-l+i2xp7;.-("liL+pj:0-L2«"l+2i^^^^^ 

■Hj)U*-2i6)(/,o . >fjAi/-HVo . <^/-C-lpL-(.( (Jcopj) /(.w/j/.;;^ 
99 '99 +.x'()«o.=7),o] 
[2] p',7/.rs PRGGRA:; COliVUnTS the algebraic STRIllG TO POLISH iJOTATTOn 
[3] (\THIS PROGRA:": DEVEWPED BY WILEY GRIHER 
V 

VC/lP5[ni7 
V CAPS XZ;K;LCn;0;Hl;n2;H2 

[I] ->-10xi~23=f/,Op(2-^iO 
[2] ->-llxi{5[i3s« » 
[3] iniTTAL 1 

[4] -»-6xi;;'=/)r) 0 

[5] G0!'A{hCt<lyTVll-i-2i^ 60 60 60 60 Tl20] 

[6] SIW 

C7] " 

[8] fv'-«-"23 

[9] SAVEi'fESSAGS 

[10] -0,p[>'.4f.L r//r; PROBLENS III THIS SET HAVE BEE.'! AIlSvJERED, COME BACK 
miXT WEEK UUEil THERE IS A IIEII SET OF CAPS , • 

[II] -'•!vxi(i29)=10i~l+'0123U56789' iO[2 3 4 5~6 7] 

[12] 'UARHTUCh YOU ARE HOT SIGrlED OH TO YOUR PROPER HUOBER' 
ri3] • 

PLEASE CALL THE LAB ASSISTANT IfS-fEDIATELY I ' 

[W] 

7 
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7 S /-^ A POLISH I;L 

fl] i'-(('--Z'€'()')/I)ra.,^(2,pL)cL,(~f/£L)/>"-l+i2xpi,^(-l+i + VpL)-L2® 



7 /I ///I L YSIS ; I/;5 ; ^ ; ; r ; 5 2" ; P 

20)-i',0pj/v.'J-:';.Or'/-(i20),op-'V'(. '] ,opi:-(i 
2 ] 1 , (2->-(2 , ( 6- vrr T),' i: ' , J;V. ■ 

■ 3 ] -^/l ^^i" I : = P I ) A f ' * ' = ~ 1 + J A-.s- ) A ( 1 + J/; c ) e 2 7 1 /: Z, r 
r '« J ■)-2 X I /I i Cy.Wr /' J/V. 7 

- .1 2^0 , ( c vj;y T ) ,iv!^^inF: ,(0 = 0 r//.9 ) / » 0 ' 

6 3 SETtiur. 

17.] P-t-32)p0 

ra] GOKVAL PFf-A POLLS it ^i"^ 

01 -1 

. to] At-oz-^lUxil'AL^ALPxliinin^^VALllI 

Cll] CA'M-f ?8 ; ] ; ' , 70(7 COT THAT chr' 
12] ' 



1 1 3 ] --0 , qo-^o , ( cvrr-i t ) , i/.'j ,»({)' 

.C lU ] H///ll'r ?C ; ] ; ' , rO.'.' /'J5r,TZ? T.'/J^ . ' 

15] -»-23xii = /;/W/P+ 1 

16] O-i-O , ( C VTl': r ) , ' ? ' , JiV.-^ 

fl7] -»-( 0 18 20 22 )[///'] 

'IC] 'LE-rS RE EE AD THE PRO:' LEU.' 

IS] -^l.pijW/l 

L20] '£,r)C?A-' 

'"21] -^1,plJ^/72 

, 22J /;3 

L 2 3 ] 0-^0 , (cm- r ) , , t y\ f 

r2u] ' 



v,vE.vLjfrr']y 

! V Z-^NEWLIT A;U 

11] 2-^(~l*lp/l€ )x(10*l+5-((t-./l) I ' . ' )x 101-1+ 10123456709 ' i (/I e' 0 12 34 56 789 ' 
)//l-<-/I,(~' . 'eA-^Cl+A I '£".)p/1 )p ' . ' ,OpZ?^{-l*lpre '~- • )x I0i~l + ' 012 34 55789 ' 
i( Ip-'Je' - ' I T ')+/!, (2x~«E'e/j^(/ic '01 234 56789. )//J )p'00 ' 



V 
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V PBESEHT X;I;L\^;J;V 

f^] I'^I^H'I'fJ^- ';^)+^-/2<2(^+l).,Y3)i5.0pLC//W/W2-;/3-,' • 

[33 l^/liri;i^-/y£-;/Lly /"l + (;>'/i/7)il/,Cpl^^/,^32pOxJ^2 

CS] -^((0 = pi^),(.;i'p^'4' = i^l^^;r+l^))/ > r 

do] //3^ 1 F 



^ PARMA. ".'/[. "I ]V 

V Z^PARMm. CP ST;K;L 

Cn ■^OxiA/o=;^-i-(,'7:/'o. = i() t ) + J -J 

f2.1 ->-5x i(.Jf>pZ)v/r<£,^( • f AVZ I'DtZ) X 1 

. [3] ■*2yx~A/0=z,0f-iZ[ (1+K-L) ,K:i^O 

] -vo 

;5] SW^v/MV 

C6.1 'PARSIITHESIS HISSl^AfC" • SW/'ArJD' 

"73 zr]^o 

fl3 -^(2-!-l26)xi;:<pZ 

[93 -•■0,2[l+A-i]^i 

•■10 3 2[ia-l 
V 



7 f/i4L £•;:[{": 3? 

^33 QU+xpQ:\^0^O,(.':vN' T) ,TVr.l + 2i 60 60 60 60 Tl203 

I 53 SAVSnESSAOE 

^ [ C 3 1 1 I ( 2 6 = p L C /; ) V ( 1 0 0 < p 0 ) V 3 2 < + / r ,v C e /, L n 

I 7 -^Oxi^^A/r/^mVeLC.V, 't-xfC )0l23'i 50709'. *• 

[93 (~'/)\' + ' ■' " ' 

:'"103 ->-o,pn-^i/.'5 

i 113 'you ARE PLAUne IflTJ; my. BUTTCHS AUD hot WORKT^T TPE PpnDL'^t.n 

n33 li^ir. '''' '^^^ //S 

{ 114 3 ->-16xi///? 

.153 -^3,f.^[l]^'5' 

[163 ->-(17 18)r l + Oa;Vp] 

rjl^ THERE ARE PO RIHTS FOP TRIG PROBLEM' 

i 10 J /,'^2 



The CATS -^rograM iuis no;; been o]^ot:at:io:Ml for throo years, accumnuilnting 
a total of. cODO on linc^ stuclcal- iiours • The progran 3s ur.ed almost c>:clusivcly 
by stiKlcnts i r.rollcd in Math 003, ti^o remedial iiiathunatics course at both 
Golden Vest and Oranf^e Coast CoJlogt-G. 

There arc eight units in thi: CAPS program corrr^sponding to the eight 
chapters in the Math 005 syllabus, Kach unit consists o£ twenty (20) word 
problems and dja^s with t-he subject uatter presented in class and on the audic- 
tutori-jl and viuco tapes. Thus all media are synchronised to present the same 
concepts over the duration of two weeks, 

A student who is ready to do tlie CAPS program Cor any one unit reports 
to the iMulti-Mcdia Center or the computer room in the Math-Science Building, 
signs on to the computer and calls for the program appropriate to that unit. 
The interaction for any given proble;n is illustrated in Figure 2, Student 
responses are always preceded by the symbol "n"* called "quad," Everything 
else is computer generated. In the first problem of Figure 2, the computer 
typed everything up to the first quad. The student then typed B - C and 
returned the carriage, giving control back to the computer. The latter in 
turn responded with D^-Z^O, The student, in turn, typed C ^ B, This time 
the computer came back with E f-, 2222345686, The student typed E x 100; the 
computer responded with F^ 22, 22345686, Note that the computer does not comntit 
itself to recognize the answer; the student must command it to recognize the 
answer by typing a single letter followed by an asterisk (F*) , 

The forte of the CAPS program is its ability to trap student responses and 

-as- 



m2 



IS 

A CALCOLO UOO DSSK CALCULATOR HAS 3ESII REDUCED FROM 179.99 
TO 139.99 DOLLARS. WHAT IS THE PERCENT CF DISCOUNT? 



ERJC 





UOO 


8^ 


1 79 .99 




1 O y » y 3 








H U 


Li U-^D 






0 ^ 2 222 o*45o 85 


n F>c. 10 0 

U C/ J. V 






22.22345686 


□ F* 




CORRECT 
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f/.VZ? U7 PERCENT 


Or 3i4. 


A-^ 


0 .47 






□ /Ixi? 






15.98 


□ C* 




fJ/FSr RATE, YOU 


GOT THAT ONE 



17 

.V//42* IS .25 PERCENT OF .015? 

4-*- 0 .0025 

D-*- 0.015 

□ /IxB 

C-^ 3.7 5ff~5 

G C* 

K/?/?/ COOZ? , YOU GOT THAT ONE 



ID 

37 PERCENT OF WHAT NUMBER IS 61? 

A-*- 0.37 

B-*- 61 

.C-H 0 

0 A^B 

D-*- 2 2.57 

(3 Z?* 

NEGATIVE, YOU MISSED THIS ONE. 



save tliom for analyrif^ by still oLhor functions in tliG software* package. 
For example, if tho Instructor wLshos to see iiow student I performed on 
problem J, the dyadic function "HRKAKDOU^^" givos him this information, as 
illustrated in Figure 3, where 1=1 and J=15, The computer immediately typ 
the information upon receiving the command 1 BREAKDOWN 15," 



^ 1 BREAKDOWN 15 

r*^)- 3 STEP 1 88 SEC B^C 

STEP 2 118 SEC B-C 

STEP 3 31 SEC ^ E^B 

STEP U 13 SEC F 

TOTAL TIME 25 0 SEC CORRECT 



The information in Figure 3 can be read as follows: "On problem //15, student 
#1 took 88 seconds to decide on his first step toward his solution; that step 
was B -^ C. (The computer assigned this result to the letter D, in other words, 
D is the result of B -i- C.) After 118 seconds, the student typed B - C 
(seemingly he changed his strategy), and the computer assigned this number to E. 
It took 31 seconds for the student to command the computer to calculate E B , 
which the computer called F. Finally, it took 13 seconds for the student to 
decide F was the correct ansx^er: He typed F*. Total time: 250 seconds, and 
the answer was correct." 

This is but one of the varied analyc;is functions performed by the CAPS program. 
As an example, consider the display in Figure 4. By typing "ANALYSIS 23" the 
instructor is able to item analyze problem //23 in quite a lot of detail. This 
function first types the number and responses of students who answered question 
//23 incorrectly. Then it types the entire groups' responses. The time vectors 
(these are two digit symbols in BASE 60) are not translated by this function. 
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USSTIOl'J 

• 00 2 
500M 

ne 

5051 

' 5000 
5001 
5002 

: soon 

5011 
5012 
5017 
5018 
5026 
5029 
5030 
5031 
5037 

• SOU 2 
50UU 
5050 

• 5051 
' 5052 

5053 
; 5054 
5057 
5060 
5126 
5065 
5066 
5067 
5070 
5071 
507i» 
5081 
5100 
5128 
5137 
51 no 
5142 
OK? /ES 



23 




2 
2 

""l 

2 
1 
2 
1 

0 
2 

2 

2 

2 

2 

3 

3 

1 

0 

1 

3 

2 

3 

1 

1 

2 

2 

2 

i 

2 

1 

2 

2 

1 

2 

6 

2 



2 
2 
0 
2 
2 
2 
2 
0 

1 

o 

2 

2 

2 

3 

2 

0 

0 

i 

3 

2 

3 

2 

1 

2 

2 

2 

0 

2 

0 

2 

2 

0 

2 

6 

2 



12 
16 
0 
61 
23 
13 
10 
19 
33 
2 7 
29 
12 
12 
23 
29 
24 
. 34 
16 
52 
20 
12 
13 
35 
42 
7 
7 
43 
64 
50 
37 
15 
18 
12 
48 
8 
ComP 



35 
28 
0 
95 
43 
28 
15 
19 
50 
49 
46 
32 
21 
38 
78 
32 
34 
20 
83 
36 
30 
18 
38 
57 
24 
21 
49 
79 
54 
53 
35 
22 
30 
264 
20 



2 
2 
0 
2 
2 
2 
2 
0 
1 
2 
2 
2 
2 
2 
2 
0 
0 
1 
2 
2 
3 

2 

1 

2 

2 

2 

0 

2 

0 

2 

2 

0 

2 

2 

2 



1 

1 

0 

1 
1 

1 

2 

5 

2 

1 

1 

1 

1 

1 

1 

5 

5 

2 

1 

1 

3 

1 

2 

1 

1 

1 

5 

1 

5 

1 

1 

5 

1 

2 

1 

FifiiT Of 
t-X.T 

I I 34 



3oi'+Cc _/J-Z?c ,/"A5C;^/v*v/v0l4'» 

c//;-i? + CcT/M 
coCA 

c I UOc ]/;-/! c/£-tD 
cpi; + 6'eA'Z?-/.c| 

cp5 + Cc ,Z?-/lc ;Oc~£'tD 

cf 15c:Z?tD 

cOZJA 

c*U0-25c uZ^t> 
c^l5c| D\i 

cIB-Ac ,A-Cr\9>D+EcDAi-Bc iB-i-C^JiH-A 
c iB + CciD-AciEp 



QUESTION 23 



,1 

b.. 60 075 
1 . 8529'11176 
S ) 




1582 29 5U 
1. 76U705882 




76470588 45.52941176 0.8529411765 1.588235294 



Thus, 11,0 yt.-pH C.r student. 500/. woro: H + C ; /.. - i); .,„d F.: Ho. did not 
.-.nswoi- this q.H.stlon correctly. Stmhuu 5000, on tlio olhor h,md, answered 
correctly and typod : U + C ; 1) - A; .md E. This function further gives the 
totals, averages, and other descriptive statistics not shown in Figure 4. 
If the instructor wishes to find out how any one student has performed, he 
may typo " FIND J " , (See Figure 5.) This analysis "cross-sections" the 
problems . 

The CAl'S program contains other functions designed for even closer analysis, 
but time and space do not allow a full discussion here. 
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} 



FItW 1 

c>/lx3=>>CA 
co/?^.ylnA/ixBciZ?4) 
c(/7x/, crc<) 

=>/i. 005cA+Sc]r:^Cc€l35r(?Pc3F/iS^C . Vfv. voi 

C(j/lx/lc I 5x5c ;Z? + EcCCxFcT(7A 
35:>*D]i4xBcAylxB 



I 555001» : 

cp>lxSc3/?+Cc I Z^ii 
cB/4x3c/J + (7ciZ?A 

C.4C, lOc^/lxCc :/l+Z?cii;4 
3c>lxBc]C<}> 

cP>l x5 c //I t Cc VZ? -/I c j?<l) 
c)/Ix5ctCA 

c \ /I + Sc <x» X /> c C Z>+£'c tF4 
:>(!1.4xCc-/lxZ)c"Cx;^c ,F-Gc^n^ 
c\A>^BooA -Z)c33 0cCi?vFc | Go- 

c_x5c\Co 



FIi7Z? 3 
I 55002]o 
c^xBc/Ai-Cc;D(i 
n=>Cx/lc>^C-c2ff/. 
c\5x/lc©674> 
cIA^BctC^ 

Uli4x , i0c6Ji4 + Cc'"Z)A 
cCBx/1cCC<J> 
=>:/lxBc3C<i> 
n[>lxuc(c 



I 
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Examples of Computer Assisted Instruction Programs At 
The Coast Community Collcige District 
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THE SKGALIA MODEL FOR REIATTNG 
THE COMPUTER TO Mr\THI-3^L\TICS INSTRUCTION 



Both Orange Coast College and Golden West College have integrated 
APL into the mathematics curriculum following a model developed by 
Professor Angelo Segalla. 



The 

^^y"^^ computer as 


ANALYSIS 


The computer as a 


CALCULUS 


tool to aid the ^^x^ 
discovery of 
nia c h ema t i c^^^^ 


ALGEBRA 


^^^n instructional 
device. 


ARITHMETIC 


(APUx^ 



As a student advances through the mathematics curriculum, the use 
of CAI segments diminishes, but student use of the computer as a tool, 
increases. Special classes have been added to the curriculum which 
parallel the classes in Algebra and Calculus. These classes introduce 
the student to the CAI segments available for the subject. And, also 
to techniques of progranrlrig in APL. 

Using this model, it is possible for the instructors to concentrate 
their CAI developments at the lower levels of mathematics where most of 
the students are; and, where the best chance of cost effectiveness is. 
Still, they arc able to provide enrichment opportunities for more 
advanced students. 

It is hoped that with instruction in APL programing, the student will 
be able to "discover" mathematics. The discovery method, while 
theoretically sound, has in general failed, due to the computational 
work involved. The student who has mastered APL will find that calcu- 
lations are no longer a hinderance and should, in a s:/mbiotic relationship 
with the computer, actually be able to do some real discovery oriented 
learning. 
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The attached terminal printouts represent simulated Student Terminal 
Sessions. They provide examples of the variety of CAI segments used in 
the Coast Community College District. Each Segment presupposes some 
advance preparation on the part of the student. Most of the printouts 
do not include the initial interaction in which the student may obtain 
specific instruction on the control of the segment. In each segment 
the student input is underlined. 

MEDTERMS 

A drill-and-practice segment dealing with medical terminology and the 
meanings of commonly used medical prefixes. Questions and choices may 
be presented randomly. A retort is given for each student choice. A 
series of ^0 similar segments have been developed which use a computer- 
driven Microfiche Projector. 

SPANISH 

A comprehensive series of instructional segments covering the whole of 
Spanish grammar, including the formation of all verb tenses; gender, 
plural and uses of nouns, pronouns, adjectives and articles; uses of 
adverbs, prepositions, and verbs; language syntax and other items related 
to Spanish sentence strategies. The series includes a 3,000 word 
vocabulary and approximately 200 idioms. The organizational format 
follows that of Foundation Course in Spanish" by Turk and Espinosa. 
n nnnf^J?^ segments providing 100 hours of instruction. Eleven hundred 
(1,100) student hours Resulted from 1,200 student contacts during the 
Fall semester. ^ 

MODIFIERS 

A tutorial program in English which requires the student to find and 
reposition a misplaced modifier in a sentence. There are 30 segments 
providing 21 hours of English instruction in such areas as spelling, 
paragraph coherence, sentence fragments, modifiers, transitions and 
2 ^^^v^" hundred (1,100) student hours resulted 

from 1,300 student contacts during .the Fall semester. 

PROBLEM 

One of a series of segments which present word problems, and allows 
the student several options in its solution. The student may supply the 
solution, request more data, or enter an analysis mode in which he is 
asked to perform in a manner similar to what he would in an office 
session with an instructor. For this example the instructions, a problem 
?S c.M-L^Mf% analysis steps are shown. The software for this segment 
iord JJobleml?" Chemistry, or other subjects using 

LIMIT 

A segment designed to help a Calculus student with delta-epsilon limit 
proofs for polynomial functions. This is the first of a series of 
segments devoted to limits and limit proofs. The student has the option 
in this segment of requesting the proof and an explanation of any step 
in the proof. The student inputs the coefficients of the polynomial of 
his choice and the point where the limit is to be evaluated. 
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SUPERGRAPH 

One of a series of plotting routines designed to aid the student in 
the discovery of mathematical relationships. The student may choose 
the function to be plotted and the domain of evaluation. Included in 
this workspace are routines to provide polar and parametric plots as 
well as routines to locate the real zeros of the function. 

ZPLOT 

A segment which visually displays complex roots and multiple roots. The 
student may use standard textbook notation for defining the function 
By observing the function as the junction of various regions, the zeros 
may be obtained. In this example, a multiple root occurs at 4. real 
roots occur at -3.1 and -0.8, and complex roots of (1+1. 5i) and (4.7+3 51) 
Sf accuracy workspace will locate the zero? to a higher dlgr^ee 

LEASTFIT 

A physics workspace used for analysis of experimental data. In this 
example the student receives both descriptive statistics and a histo- 
gram comparing his data with a theoretical Gaussian Distribution having 
the same mean and standard deviation. The * printed in this Histogram 
represents points through which the normal curve would pass. Included 
in this workspace are other curve-fitting and analysis routines. 

ELECTRONICS 

A simulation segment which provides the student an opportunity to 
trouble-shoot a randomly generated failure In a given circuit. The 
student is given the symptoms of the failure and has the opportunity 
to test and/or replace various components at his descresion. The 
trouble persists until the failing component is replaced 



miCH PBEFIX- hEAm 'HEART'? 



CEPHAL- 
CONTRA- 
CABDIA- 
CHOL- 



03: CARDIA 
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CORRECT-THAT WAS CLOSE\ AS WED IN QMMPGRAM WHICH IS A WACING OF HFAPT 
RHYTHM AND PATTERN. PROCEED TO NEXT^EM. -BALING OF HEART 

5. miCH OF THE FOLLOWING PREFIXES DENOTES A 'BLOOD \'ESSEL'? 

CEFflAL' 

CHOLE- 

BRADY- 

ANA- 

ANGIO- 

Q: ANA 
Q: BRADY 

TRY^fml ^ mm.CABDIA WICH MANS SLOW HEART. 

Q: MA 

YOU HAVE MADE THIS CHOICE BEFORE 
Q: MGIO 

6. WHICH PREFIX MEANS 'AGAINST'? 

BRADY- 

CON- 

BI- 

ANTI- 

CLEIDO- 

CONTRA 

YOU MUST CHOOSE ONE OF THE ABOVE CHOICES 
Q: ANTI 

l%:j^T(i0NTfNm. "^^^ ^ SUBSTANCE AGAINST VACTERIAL LIFE. 

7. WHICH PREFIX MEANS 'CARTILAGE'?-^ 

CEPHAL- 
OSTEO- 

CARDIA- 
CHOL- 

-CHom nA ^ - - 



PLEASE NOTE 



° 2?/^ SMALL CIRCLE IS USED TO INDICATE TNCORRKCT SPELLING 

EX.: TRIAL MISPELLED = TRI^L, 

^ TllE ARROW POINTS TO INCORRECT PUNCTUATION. 

EX.: m LLAHO PEPS^ (NO PERIOD). 

- UNDERLINE LETTERS THAT NEED AN ACCENT OR TILDE 

'BX.: LiPIZ mSAM 

miiai SECTION DO YOU WANT?. . . 5 

GIVE IN SPANISH. DON'T USE THE SUBJECT PRONOUNS. 

WE SPEND A SHORT TIME HERE EVERY DAY. 

GASTAWS UN ????? AO.UI TODOS LOS DTAS. 
oASoooo UN 0000 /iQUo TODOS LOS D^AS. 

WE SPEND A SHORT TIME HERE EVERY DAY. 

PASAWS UN RATO AQ UI TODOS LOS DIAS . 
'.ESQ EST = 
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(mT TIME IS IT? 

QUE. HORA ES? 
iQUE HORA ES? 

MAT TIME IS IT? 
?... ?QUE HORA ES? 
THAT'S CORRECfi 

I HAVE A LOT OF TIKE TO STUDY. 
?... HELP 

DICK, HOW ABOUT: 

TENGO MUCHO TIEMPO PARA ESTUDIAR. 



I'M NOT GOING WITH M MAMA THIS TIME. 

■ NO voY CON m mm est a vez. 

\ BRA VlSIMOl - 

I GO TWO Tims E\rERY WEEK. 

VOY DOS TIEMPOS CADA SEMAfJA. 
VOY DOS 0000 0 CADA SEMANA. 

I CrO TWO TI14ES EVERY WEEK. 

VOY DOS VECES CADA SEMAN A. 
\ESO ESI — 



DO YOU ALWAYS ARRIVE ON TIME? 
?... STOP 

OK DICK. WE'LL STOP HERE FOR NOW. 

DO YOU WAllT TO SEE THE SCORE BOARD?... SI 



LESSON 5 
TIME If 

PROBLEMS TRIED 5 
ANSWERED CORRECT LY _ i{. 



TOMORROW MAIANA 

mn hVY 

MANY mCHO 

** ANSWER: MANY-WCHOS 

NORTH AMERICAN AMERICANO 
NORTH AMERICAN NORTE AMERICANO 



IN THE AFTERNOON {HOUR GIVEN) 
DE LA TARDE 

14UCH0S 



TIME 


LA 


VEZ 


TIME 


LA 


HORA 



CHARLES CARLO 

aiARLES CARLO 
-k-k ANSWER : CHARLES -CARLOS 

URUGUAY EL URUGUAY 

THERE IS HAY 

CHARLES CARLOS 

SON STOP 



ocn .,uu.u .UT SOLD TO STUD f NTS CONTAINING MORE WAN 3.2 PERCENT AIWllOL. 

TYPE TIIE FIESi 
0 

SOLD STUDENTS 



^PE TIIE FIRST AND LAST WORDS OF TIIE MISPLACED MODIFIER. 



NO, DICK, YOU HAVEN'T GOT TIIE RIGHT MODIFI^^R ***** 
TRY AGAIN. 

o 

CONTAINING ALCOHOL 



GOOD, YOU HAVE IDENTIFIED THE MODIFIER. 



miERE SHOULD 'CONTAINING MORE THAN Z. 2 PERCENT ALCOHOL' 
BE POSITIONED IN THE SENTENCE^ ALCOHOL 
D 

BEERA 



SHE WORE A RTBBON IN HER HAIR MICH WAS A LIGHT PINK. 
TYPE THE FIRST AND LAST WORDS OF THE MISPLACED MODIFIER. 
WHICH PINK 



GOOD, YOU HAVE IDENTIFIED THE MODIFIER. 



WHERE SHOUI,D 'WHICH WAS A LIGHT PINK' 

BE POSITIONED IN THE SENTENCE^ 

C 

SHEA 



SHE WHICH WAS A LIGHT PINK WORE A RIBBON IN HER HAIR. 

l^-^J^^n* PROBLEM WITH THE SENTENCE IS WAT HER HAIR APPEARS TO BP ^'TOfl'r 
PINK' WHEN THIS SHOULD BE A DESCRIPTION OF WHAT? TRY ACm 

WHERE SHOULD 'WHICH WAS A LIGHT PINK' 

BE POSITIONED IN THE SENTENCE? 

0 

RIBBONa 



ERIC 



m mRE.A -RIBm mCH- .was A .MGHr -mm: ^N::rwp,:n-Aa:n- 



)J 1 0.5 1 

051 )'~2i.'ui;rr9"'io/20/72 PFilOCC 

4 P L \ 3 7 0 

)LOAD 1231 PROBLEM 
SAVED 21.35.51 10/20/72 

GO 

IN THIS WORKSPACE YOU WILL BE PRESENTED A PROBLEM. YOU MAY CHOOSE TO 
ANSWER THE PROBLEM DIRECTLY. OR BE ASKED KEY QUESTIONS ABOUT IT 

?n'IIlTngThI PRolllT ''''' '''''''' 

^2^^U9.^ ^ODE TYPE THE LETTER S FOLLOWED BY THE NUMBER THAT YOU 

BELIEVE TO BE THE SOLUTION OF THE PROBLEM. 

^iUll ^19.^.1 TYPE THE LETTER H AND I WILL PROVIDE YOU WITH A HINT 

ABOUT THE SOLUTION OF THE PROBLEM. IF YOU TYPE HS 
I WILL PROVIDE YOU WITH THE SOLUTION, BUT THIS WILL COST 
YOU 100 POINTS. 

INQUIRY MODE TYPE THE LETTER I FOLLOWED BY ONE OF THE LETTERS 

BELOW TO OBTAIN THE FOLLOWING INFORMATION: 

D -- DEFINITION OF TERMS USED IN THE PROBLEM 

DD -- DIRECT DATA INVOLVED IN THE PROBLEM 

ID -- INDIRECT DATA INVOLVED IN THE PROBLEM 

E — THE EQUATION FOR THE SOLUTION 

C — THE CONCEPT INVOLVED IN THE PROBLEM 

U — THE UNKNOWN INVOLVED IN THE PROBLEM 

F ANY SPECIAL FORMULAE NEEDED FOR THE SOLUTION 

iEi^lill ^2.^1 TYPE THE LETTER A AND I WILL ASK YOU THE QUESTIONS 

YOU SHOULD ASK YOURSELF AS YOU ATTEMPT TO SOLVE THE 
PROBLEM 

YOU MAY REQUEST THE CALCULATOR MODE BY TYPING THE LETTER C 

THE CALCULATOR MODE ASSISTS YOU IN YOUR CALCULATIONS. TO EXIT THIS MODE 

YOU SHOULD TYPE THE WORD END ^ - lu ilah iHlb MODE 

ivL^li^///rn^rnl^^^^^ JOINTS. WHEN YOU ASK FOR ANY MODE OTHER 

CALCULATOR MODE, I MAY SUBTRACT POINTS FOR THE HELP 
MY QUESTIONS cIrIIItLY^.^^^^^ ^^^^ ^ SUBTRACT POINTS IF YOU ANSWER 

YOUR SOLUTION MUST BE CORRECT IN THE SOLUTION MODE BEFORE YOU ARE ALLOWED 
TO ADVANCE TO THE NEXT PROBLEM. ^'^'^unc xuu AHt /KLLUWbU 

TO STOP THIS PROGRAM, TYPE THE WORD STOP 



GOOD LUCK ON YOUR PROBLEMS'. 



itOl:' y-Atr/ GALL^HF OF PPA^TY rOVTAHUPr. no ^FPCFt^T ALCOHOL 
(30 PPOOF) '\'>OULD PF. ADD^P TO 100 GALLONS OF Wjrir r ON-r AT F I 0 
12 PERCEt:? ALCUOL TO PRODUCE An IFTOXTCATItlC, r>Pifpvflr,F' ' 
'^HTC'i HAS 20 P^PC.EFT ALCOPOL do ^>P00^)? 

W/'TA' TPE SYMBOL @ APPEARS YOU MAY ^FTPP YOUR MODE rpoicv r vjrr 
ALWAYS RETURN TO THE I'ODF C^'OICF STATUS FACP TIM'^. - - ■ ■ 

d 

IF X IS THE ^'UMB^P OF GALLONS OP ^RAUPY, 'r'PITE AfJ E XPPES<=; tq^J rrivniVinG 
X yPIGH IS FOVIV ALEUT TO T'lE ''UM^rp OR GALLO^^'! OF ALCOHOL J-7 ^'//T PRANDY. 

BULL'' GOOD 
A 

HOW MAtiy GALLO'lS OF ALCOHOL AR'^ If,' THP ''TNE? 
12 

YOU ARE CORRECT OU THIS PORTIOP 0^ TIE AtJAL^SIS 
-A 

RFIIEMbVrIHG that X IS THE HV'iPPR OF GALLOPS OF BPAflDY , WpITE AH EXPRESSION 
IHVOLVIHG X ypiCH EXPRESSED '^HF IIUMPEP OF GALLONS OF 'T'^p RESULTING 
BEVERAGE. 

A'+lOO 

VERY GOOD 
J5 

A 

^P.ITE AH EXPRESSION , INVOLVING X, WHICH '^XP^-FS'^'^n T''E AMOUNT OF 
ALCOHOL IN T'F RESULT J^'G VEVERAGE . 

.2x(ina + X) 

V'-^HY GOOD 
i?] 

.. J 

'■■/piTE AH EXPRESS rnrj , INVOLVIHG X, h'Hinil TS EOUTVALPHT TO 'PR^ 
'WMPFP OP GALLONS OF ALCOHOL TH THE PR ANDY PLUS fRP NUMBER OR 
C.ALLONS OF ALCdOL IN THF WJP^, 

'^+it+2x . 2X 
EXCELLENT 

■ _ A 

Wf'khow the TOTAL NUMBER OF GALLONS OF ALCOHOL IF THF BRANDY 
AHD IN THF 1-/TNP ADDED TOGETHER TS EQUAL TO THE TOTAL NUMBER OF 
GALLONS OF ALCOHOL IN THE RE'^UL''^inG BEVERAGE. UPITE /JV EQUATION 
lUVOLVTPG X, WHICH EXPPF'^'^ED THJ'^ FACT. * 

12 + .UX = .2x(j?+100) 

'^xaU:FLjEChU:Am:T:ClNL=U:ASi:LT^^^^ 



LIMT? 



THIS FUtWTION IS DESIGNED TO ASSIST YOU J.W V'^ITIVG 'A e' '^"OCS FOR 
POLynOMIALS . WHEN 'F(X)' APPEARS EtlTEF T!l^ COF'^FirTEi'TP THE 
POLYUOI'.IAL. WUEll ' A" -»■ ' APPEARS EtlTF.P ?H" POIV? THE Dr>\iATfl W."ERE 
THE LIl^IT or THE FVUCTIOU IS TO 3E EVALUATED. 



FiX) = 
□ : 

1 J) ~2 U 2 

H: 

2 



Lit-! - 2;:*2 + 4/ + 2 = 18 

CHOOSE A < e / 59 WllEf! 0 < e < 59 PO VO'! U'AtJ? P^OOP? YES 
VHEU: c>0; 0<A<1; F(A')=/--it - 2X*2 + i»/ + 2 

/I = 2; /. = 10 HERE IS HOW TO FIND A ^ AS A ^DNCTIOfl 0^^ ^. h<^( c) 

SUCH THAT 

IF 0<\X - A\<^ THEN \F(X) - L\<e 

1) 0 < IX - 2| < A ; U*it - 2;:*2 + UX + 2 - 18 I < s 

2) 1 < X < 3 ; U*it - 2/*2 + UX - 16 I < e 

3) U*3 + 2/*2 + 2.V + 8 I X 1/ - 2 I < e 

U)U - 21 < 6 / U*3 ■+ 2/*2 + 2X + 81 

CHOOSE A < 6 / 59 f///i7/l7 0 < < 59 
ENTER 0 r7P LINE NUMBER YOU WISH CLARIFIED 

J// 5>ffP" 3 WE ARE EXPRESSING 

\F(X)-L\ OR \X*H - 21*2 + HX - 16|, AS THE PRODUCT OF 

\X-A\ OR \X - 2 I WHICH MUST FACTOR EVENLY IHTO \F{X)-L\, AND ITS 

COFACTOR: I ( F( X )-/.)/ (/-/l ) | OR \X*Z + 2^*2 + 2Z + 8| 

ENTER 0 OR THE LINE NUMBER YOU WISH CLARIFIED 
1: 

2 

IN S'TEP"2' WE ARE DETERMINING 

THE DOMAIN OF X ; THE CONSTANT IN F{X)-L=2-1B=~1& 

ENTER 0 OR THE LINE NUMBER YOU WISH CLARIFIED 
0: 



SUPERGRAPH 
i::3TRUCTI0NS? YES 



YOU MAY Eh'TSR ANY EXPRESSION USING + - x * * | siN COS TA^i 
AND I WILL PRODUCE A GRAPH OF THE EXPRESSION. STANDARD RULES 
OF ALGEBRA ARE USED TO DETERMINE THE ORDER OF EVALUATION . 

F(X) = . 5x I 2X+ I 2X+H I I 

FOR WHAT VALUES OF X'ARE WE TO EVALUATE THE FUNCTION 
D: 

STEP ~U U . 5 



10.0 



7.5 



5.0 



2. 5 



0.0 



I 

'2 



DO YOU WANT A TABLE OF THE X AND F(X) VALUES? YES 



n 


2 


"3.5 


2 


"3 


2 


"2.5 


2 


"2 


2 


1.5 


1 


1 


0 


0.5 


1 


0 


2 


0.5 


3 


1 




1.5 


5 


2 


6 


2.5 


7 


3 


8 


3.5 


9 


«♦ 


10 



)LOAD 123U ZPLOT 



SAVED 20 .01 . 58 ll/li;/72 



1 
f 




ZPLOT 

F(X) = (EXP X - X SIN X)(X*2 -8X + 16) 
WHAT IS THE DOMAIN OF THE REAL VALUES? 

□ : 

STEP "5 6.2 
WHAT IS THE DOMAIN OF THE IMAGINARY VALUES'' 

□ : 

STEP "2 5 .25 



/ 

/ 

/ 

//--- 
11--- 
//--- 
///-- 
///-- 
///-- 
I Hi- 
ll II- 

lllll 

///// 

////// 

////// 

mil II 

llllllll—- 
II II II II--- 
IIIIIIIII-- 

iiiiiniii- 



WWWI I I I I Mill II \\\\\| I I I 

\\\\\\\ ////// WWMIII 

-WWWIIIIII///// wwwiii 

-WWWI I I I I I///// WWWI I I 

-WWWIIIIII///// WWMII 

--WWWIIIII////// WWMII 

"WWWI I I I II///// \\\\\| I I 

■-WWWIIIIII/////- 
•--WWWI I I II/////-- 
--WWWI I I I I !/////- 
---WWWI I I I \lllll- 
■---WWW I I I I I Ml II- 

WWWI I I I Mlill 

WWWI II I I Mlll- 



---\\\\\l 1 1 
•--\\\\\l 1 1 
•--\\\\\l I i 

■--WW I I I I 
■--WW! I I I 
WW I I I I 
•W\l I I I I 



Mini— 

MIIII-- 
Ml Ill- 
Mil II— 
\lllll-- 
Mlllll- 



-- -.WW 
--WW I 
-WWII 
-\\\ I I I 
-Willi 
\l I I i ! 



M 1 1 1 1 1 1 1 1 IM 1 1 1 
M 1 1 1 1 1 1 1 1 i 1 1 1 1 1 
M 1 1 1 1 1 1 1 1 1 1 1 1 1 1 
M 1 1 1 1 1 1 1 ! 1 1 1 1 1- 

Ml III I III 

II 1 1 II 1 1 -- 

mill I 

II III I- 



\WW\lllll/////-\\\ ////// WW 

•W\\\\llllll//////\| 
•WWW\I I I I MIIIIIW 
-WWWM I I I MIIIIW 
-WXWWI I I l-l I///I I I 
-WWWWI I I I I///I I I 



I I I II/////- 
I I I II/////- 

I I I M I III I- 

I I wimii- 

I II Mill I II 



www 

--wwww 

-WWWM I 
WWI I I I I I 
\l I ! I I I I I I 



-5 I 



■3 I 



■^21 



-^1 1 



VyVVyVVVVVAAAAAAAAAAAAVVVVVVVVVVVVVVVVVVVVVVVOVVVVVVVVVV X 



III I IWW 
IIIIIXWW-- 
I I I IWWW-- 
I I I |\\\\\--- 
I I IWWW--- 
I l\\\\\\\-- 

I IWWW 

l\\\\\\\ 

+ + + 
■4. -3. -2 



/////I 

////// I 

• — -/////II 
•--/////II I 

-limn 1 1 
■-///// 1 1 II 

IIIIIIW I i\\\ 

IIIIIW I I !\\\ 

+ + + 

-1. 0. 1 



i////////iiiiiii\-///////// 
\imim\\ 1 1 1 iw — ////// 

Will I III Mill |\\\ // 

I Mil mil I! I I IWW 

ii-///////iiiii\\\\\\ 



\\--/////l I I I I |w\\\\\--. 

■iimiw 1 1 1 iwwwww 

■-IIIIIW I I I I ixwwww 

+ + + + + 

2. 3. U. 5. 6. 



It 



2 t 



{ )LOAD 12 3U LEASTFIT 

SAVSD 11.55.38 12/12/72 

^.llll.§l.J.-J6 3.;31 .3, 26 3.22 2 . 91 2.98 3. 19 3.17 3. 16 3.00 3.01 3.2i» 3.01 
^_2^ 2 .9 3 3. 1 1 .3^_12._2.9_6 3. 39 3. 20 T.io T.^^'J ."OT'F. oy 3TW^3."07~3 .12 

MASS-^Ml ,W2 



1 



G AUSS FIT 
• ENTER DATA VECTOR 

: 

. INTERVAL WIDTH 
.05 



DSTAT MASS 



5 .00 



3.75 



2.50 



1. 25 





SAMPLE SIZE 


27 




MAXIMUM 


.3.39 




MINIMUM 


2. 91 




RANGE 


0.58 


o o o 


MEAN 


3.09 


o o o 


VARIANCE 


0 .01859230769 


o o o 


STANDARD DEVIATION 


0.1363536127 


o o o 


MEAN DEVIATION 


0.1051851852 


o o o o * o 


MEDIAN 


3.09 


0:^0 0 0 0 


MODE 


3.01 3.07 3 


o o o o o o ie 


SKEWNESS 


~3.908272332ff' 


1c o o o c o o 


KURTOSIS 


2 .530063807 


ooooooooo ooo 






OOOOOOOOO 0^*0 






ooooooooo ooo 






ooooooooo ooo 







oo oo oo oooo oooooo oooooooo 
oA-oooooooooooooooooo ooA-o 
ooooooooooooooooooooooo o 
oooooo oooooooooo oooooooo 

0000*00000000000000000000000000*0 

oooooooocooooooooooooooooooooooox> 

0*0000000000000000000000000000000 * * 

* ooooooooooooooooooooooooooooooooo 

t f f f f f 

2.810 2.910 3.010 3.110 3.210 3.310 xffo 



DO YOU WANT A TABLE OF VALUES? 
INTERVAL CENTER FREQUENCY 



YES 



2.Z1 


1 


2.86 


1 


2 .91 


2 


2.96 


2 


3.01 


3 


3 .06 


H 


3.11 


5 


3.16 


2 


3. 21 


3 


3.26 


2 


3.31 


1 


3.36 


0 



EXPECTED FREQUENCY 

0.U83131923U 

0 .9660180035 

1.667516917 

2. 500U2^7U3 

3. 3073U2057 

3. 8um73135 

3.885170268 

3.UU7080891 

2.669859935 

1. 82786288U 

1.09026796 

0 . 5613267593 



ERLC 



! 



VI 

mxER/osc 
izeEs 



I' CI t / 



tWteSS SHOWN OTHERWISE 

U CAPACITORS IN UF, 
i SmORS \H OHMS. 



«cif7vm 



V3 

OCTAUDtO AMPL 
I2AV7 



V4 

AUDIO PWR AMPL 
50C5 




ERLC 



^LOAB 123'^ ELECTRON I Cfi 
SAVED 12.'+1.00 Ol/OH/12 

TROUBLE 



Z?0 YOU WANT INSTRUCTIONS? NO 

NO AUDIO ON LOW BAND 

NOISY AUDIO NEAR TOP OF HIGH BAND 

SQUELLING ON HIGH BAND 

□ : 

SIGNAL TUBE 3 PIN 5 
NO SIGNAL 

VOLTAGE TUBE 3 P IN 5 
-•^.O VOLTS 
D: 

REPLACE TUBE 3 
***T ROUBLE PERSISTS*** 

□ : 

R ESISTA NCE RES U 
"+.7^/ OHMS ' ■ 

REPLACE CAP 13 
•••TROUBLE G0NE9«» 

WANT ANOTHER PROBLEM? NO 




> 



624 1 NSTR UCT I ONAL SEGMENTS 

i N 

37 S UBJECT AREAS 



308 HOURS OF INSTRUCTION 



DEVELOPED BY 
8 5 DIFFERENT FACULTY MEMBERS 




2 0 0 0 STUDc NTS 



ACCUMULATED 

- 2 0 000 HOURS 
DURING 

2100 0 TERMINAL SESS IONS 



ERIC 



SUBJECT AREA 

ACCOUNTING 

ALGEBRA 

ANATOMY 

ARITHMETIC 

ART 

BIOLOGY 
BUSINESS LAW 
BUSINESS 
CAI 

CALCULUS 

CHEMISTRY 

COMPUTERS 

ECONOMICS 

ELECTRONICS 

ENGINEERING 

ENGLISH 

FINITE MATH 

GEOGRAPHY 

GEOLOGY 

GEOMETRY 

GERMAN 

GRAPHIC ARTS 

LIBRARY 

LOGIC 

MATH 

MUSIC 

PHOTOGRAPHY ' 

PHYSICS 

PLOTTING 

POLICE SCI. 

POLITIC SCI. 

PSYCHOLOGY 

SEC. SCI. 

SLIDE RULE 

SPANISH 

STATISTICS 

TRIGONOMETRY 



SEGMENTS 


1 1 /VlL 


8 


3:35 


37 


20:10 


20 


10:00 


A A 

44 


1 9:30 


1 


:05 


16 


6:10 


14 


9:45 


1 


1 :00 


3 


:40 


42 


1 5:05 


31 


1 1:25 


13 


7:45 


7 


8:05 


5 


2:15 


1 

1 


:50 


38 


20:30 


9 


3:20 


2 


1:00 


1 


:30 


7 


2:00 


3 


1:30 


3 


1:30 


1 


:30 


5 


12:20 


1 

1 


:20 


1 


:40 


I 


:30 


28 


1 1:25 


6 


2:40 


31 


9 :20 


17 


9:00 


3 


2:00 


8 


3:50 


14 


6:30 


193 


99:30 


8 


2:15 


1 


:?0 



WHY APL FOR CA I ? 



I. APL IS A NOTATION NOT A COMPUTER LANGUAGE. 

A, It was designed to comvey ideas in the classroom rather than drive 

A COfWnER THROUGH CALCULATIONS. 

B. The computer implenentation cljosely Foao^-^s the notation and therefore 
IS not limited by the hardware concepts of first and second generation 
Von Neumann machines. 

II. 'a PL IS THE ONLY AVAILABLE TELEPROCESSING 
SYSTEM THAT WILL SUPPORT THE CAI NEEDS OF 
THE COMMUNITY COLLEGE. 

A. DrIU. S practice SHGMEr>ITS. 

B. problem solving computatictial mode. 

C. TiTTORIAL with WLTIPLE BRANCHIhK5, 

D. Simulation 

III. APL IS THE RICHEST L A N G U A G E. A.VA I- L A B L E. 

A, There are over 70 primitive functions and 3 operators as opposed to 
less than 1/2 dozen primitives for BASIC^ FORTRAN etc. 

B. Extension of data types to TENSORS (scalars^ vectors^ mattrices^ arrays 
etc.) eliminates the need for looping in many cases A^D reduces the 
programing to simple descriptions of the problem. 

IV. APL SUPPORTS THE CREATION OF SPECIALIZED 
CAI LANGUAGES. 

A. Function call strxture allW'^s for easy communication and transfer of 
•data betveen functions. 



R, Functions vvritten for one C A I Segment mv be easily copied 

" A^© used for A^^'OTHER SEG^'lENT, 

C, Under program controu it is easy to turn the system over jo 

THE calculator MODE IF REQUESTED BY THE STUDENT, 

APL ALSO SUPPORTS AND MEETS SJUPENTS 
NEEDS FOR A PROGRAMING LANGUAGE, 
'A, The language is easy to learn, 

B, Student \am pretest each step of program before entering IT» 

C, Student is not bogged dov;n with flowcharting^ data structures^ 
looping techniques, and other trivia AMD is allowed to solve 
problems in a natural mY. 

n. Input/Output is easy to handle and requires only primitive functions. 



NORMAN £. VMTSON . CHANCELLOR 
ROBERT SCHAULIS - Director 
Information Services 

<7|4) •34.SeSS 




1403 
MODEL NT 
PRINTER 



2540 
CARD READ 
PUNCH 



2 




_ 8 




2 




1 




C 




0 




N 




T 




R 


U 


0 


N 


L 


I 




T 



2701 
CONTROL 
UNIT 



CH 1 



I 



CH 2 



HEWLETT 
PACKARD 
2114B 




T^mt COAST" CTTT?^ 



/datapoint 

( 3360 
\. CRT'S 




r 

l/oast 





IBM / 370 - 155 


B 
Y 


0/S NUCLEUS 


B 


C 
H 


T 
E 


HASP 


L 
0 

c 


A 
N 


M 
U 


ENVIRON 1 


K 


M 
11 

E 

L - 


L 

. T 


DS/3 


M 
U 
L 


1 


I 
P 


APL 


T 
I 


C 
H 


L 
E 
X 


DISTRICT BATCH 


P 
L 


A 
N 
N 


0 
R 


STUDENT BATCH 


E 
X 
0 


E 
L 




DISTRICT T & A i 

— ■ ; . [ 


R 


2 



f f 

Community Oollege district 

Information Services 
2701 FAIRVIEVY ROAD • COSTA MESA • CALIFORNIA 92626 



QOQ 



3420 TAPE UNITS 



2703 

CONTROL 

UNIT 




TIME 

DIVISION 
MULTIPLEXOR 



40.8 KB 



TIME 

DIVISION 
MULTIPLEXOR 



2741 

TERMINALS 




ORANGE COAST COLLEGE 



.bULUEN WEST doilEGF 



/datapoinm 

< 3360 1 



2741 
TERMINALS 



MICRO- 
FICHE- 
READER 



H. P 

L 
0 



REFERENCES 



CAI 

EVERYTHING YOU ALWAYS WANTED TO KNOW 
ABOUT CAI 

COMPUTER ASSISTED INSTRUCTION IN THE 
COAST COMMUNITY COLLEGE DISTRICT 

COMPUTER ASSISTED INSTRUCTION - AN 
INSTRUCTOR'S VIEW 

AN EXPEDITION INTO APL 

APL 

A PROGRAMMING LANGUAGE 
APL/360 

APL/360 REFERENCE MANUAL (2nd Ed) 
APL USER MANUAL 

APL/360 PRIMER 



Luskin, Gripp, Clark, Christiansen, 
Computer Uses in Education, P.O. Box 
1053, Huntington Beach, CA 92647 

IBM, 1973 



Anna Marie Thames, Golden West 
College, .1971 

Lewis and Clark, Orange Coast 
College, 1970 



K. Iverson, J. Wiley, 1964 

Gilman and Rose, J. Wiley, N.Y., 
1970 

Sandra Paxin, SRA, Chicago, 1972 

A.D. Falkoff, K. Iverson, IBM, N.Y., 
1968 

IBM, N.Y. , 1968 



